-- Logs begin at Fri 2022-09-16 04:53:21 UTC, end at Fri 2022-09-16 06:04:36 UTC. -- Sep 16 05:22:35.912722 dv-cinder-857528-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 16 05:22:35.920115 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 16 05:22:35 2022] *** Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: nodename: dv-cinder-857528-3-iscsi Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: machine: x86_64 Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: clock source: unix Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: pcre jit disabled Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: detected number of CPU cores: 4 Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: current working directory: / Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: detected binary path: /usr/bin/uwsgi-core Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: your processes number limit is 47680 Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: your memory page size is 4096 bytes Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: detected max file descriptor number: 2048 Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: lock engine: pthread robust mutexes Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: thunder lock: enabled Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 16 05:22:35.925615 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 16 05:22:35.963784 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: Python main interpreter initialized at 0x557cce2569c0 Sep 16 05:22:35.963784 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: python threads support enabled Sep 16 05:22:35.963784 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: your server socket listen backlog is limited to 100 connections Sep 16 05:22:35.963784 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: your mercy for graceful operations on workers is 90 seconds Sep 16 05:22:35.964149 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: mapped 403077 bytes (393 KB) for 2 cores Sep 16 05:22:35.964224 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: *** Operational MODE: preforking *** Sep 16 05:22:35.964508 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: *** uWSGI is running in multiple interpreter mode *** Sep 16 05:22:35.964508 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: spawned uWSGI master process (pid: 81543) Sep 16 05:22:35.964721 dv-cinder-857528-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 16 05:22:35.965856 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: spawned uWSGI worker 1 (pid: 81556, cores: 1) Sep 16 05:22:35.966667 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: spawned uWSGI worker 2 (pid: 81557, cores: 1) Sep 16 05:22:35.966837 dv-cinder-857528-3-iscsi devstack@keystone.service[81543]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 16 05:22:40.310785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 16 05:22:40.310785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: from cryptography.utils import int_from_bytes Sep 16 05:22:40.311539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 16 05:22:40.311539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: from cryptography.utils import int_from_bytes Sep 16 05:22:40.311948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 16 05:22:40.311948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: from cryptography.utils import int_from_bytes Sep 16 05:22:40.312068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 16 05:22:40.312068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: from cryptography.utils import int_from_bytes Sep 16 05:22:40.368625 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 16 05:22:40.368813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 16 05:22:40.368897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 16 05:22:40.368988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 16 05:22:40.369068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] command line args: [] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 16 05:22:40.369141 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] command line args: [] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 16 05:22:40.369241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 16 05:22:40.369328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 16 05:22:40.369399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ================================================================================ {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 16 05:22:40.369477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ================================================================================ {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 16 05:22:40.369764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] admin_token = **** {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.369893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] admin_token = **** {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.369964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] config_dir = [] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.370029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] config_dir = [] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.370133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.370273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.370367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] config_source = [] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.370450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] config_source = [] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.370530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] debug = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.370610 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] debug = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.370692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.370979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.370979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.371130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.371194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.371264 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.371460 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.371550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.371687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.371770 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.371910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] list_limit = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.371996 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] list_limit = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.372057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] log_config_append = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.372119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] log_config_append = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.372261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.372339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.372399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] log_dir = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.372468 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] log_dir = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.372532 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] log_file = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.372595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] log_file = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.372782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.372890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.373010 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.373099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.373254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.373344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.373497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.373575 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.373676 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.373791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.374012 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.374099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.374172 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.374243 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.374323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.374402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.374550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.374646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.374779 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.374885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.375009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.375120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.375264 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.375373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.375480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.375625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.375740 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.375926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.376113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.376264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.376430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.376557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.376703 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] publish_errors = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.376821 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] publish_errors = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.376958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.377102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.377444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.377561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.377561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.377864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.378014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.378168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.378284 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.378398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.378561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] standard_threads = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.378686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] standard_threads = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.378823 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.378935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.379032 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.379171 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.379171 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.379369 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.379465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] use_journal = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.379620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] use_journal = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.379718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] use_json = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.379853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] use_json = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.379964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] use_stderr = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.380069 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] use_stderr = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.380165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] use_syslog = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.380258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] use_syslog = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.380352 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.380446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 16 05:22:40.380557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.380661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.380757 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.380851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.380965 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.381143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.381281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.381433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.381633 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.381750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.382376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.382492 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.382611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.382727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.382848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.382964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.383078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.383201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.383317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.383437 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.383559 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.383673 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.383788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.383972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.384089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.384203 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.384323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.384443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.384556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.384682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.384798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.384912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.385026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.385145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.385259 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.385376 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.385486 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.385600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.385719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.385859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.385987 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.386114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.386229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.386342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.386461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.386576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.386722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.386857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.387022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.387147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.387262 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.387377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.387490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.387679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.387802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.387924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.388039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.388153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.388266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.388392 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.388508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.388628 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.388743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.388856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.388969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.389093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.389214 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.389336 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.389450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.389610 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.389738 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.389899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.390017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.390132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.390256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.390371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.390494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.390609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.390978 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.391095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.391208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.391325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.391435 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.391532 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.391627 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.391722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.391817 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.391949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.392062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.392160 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.392257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.392360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.392477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.392582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.392710 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.392873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.392975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.393094 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.393211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.393326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.393452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.393576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.393693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.393830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.393950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.394055 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.394167 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.394296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.394419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.394536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.394647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.394761 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.394882 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.394996 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.395111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.395232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.395349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.395464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.395577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.395577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.395800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.395916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.396029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.396143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.396266 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.396390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.396506 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.396628 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.396751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.396867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.396981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.397091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.397213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.397329 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.397444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.397549 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.397662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.397782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.397936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.398147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.398259 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.398374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.398491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.398605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.398729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.398847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.398962 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.399084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.399200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.399321 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.399513 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.399633 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.399790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.399911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.400033 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.400149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.400273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.400389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.400504 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.400619 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.400735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.400849 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.400976 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.401107 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.401227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.401343 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.401458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.401573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.401688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.401810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.401944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.402059 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.402174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.402288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.402402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.402530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.402649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.402765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.402879 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.402994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.403108 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.403234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.403351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.403465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.403580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.403702 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.403702 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.403973 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.404124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.404222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.404338 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.404437 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.404550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.404676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.404794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.404908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.405023 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.405137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.405258 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.405382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.405498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.405621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.405741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.405911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.406030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.406158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.406274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.406389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.406503 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.406617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.406731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.406855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.406970 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] auth.external = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.407115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.407244 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.407362 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.407362 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] auth.external = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.407578 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.407703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.407829 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.407829 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] auth.password = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.408048 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.408165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] auth.token = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.408789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.408913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.409038 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] auth.password = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.409155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.409271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] auth.token = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.409387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.409503 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.409626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.409781 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.409912 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.410030 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.410156 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.410264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.410369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.410469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.410566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] credential.caching = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.410662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.410767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.410865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.410969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.411070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] credential.caching = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.411167 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.411269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.411372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.411470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.411583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.411686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.411782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.411782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.411960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.412058 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.412162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.412261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.412431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.412535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.412632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.412729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.412834 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.412942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.413039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.413143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.413247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.413345 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.413520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.413621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.413725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.413879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] federation.caching = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.413985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.414083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.416774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.416888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.416888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.417125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.417205 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.417278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] federation.caching = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.417359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.417434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.417506 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.417577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.417649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.417720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.417799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.417906 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.417982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.418054 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.418126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.418198 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.418278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.418351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.418423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.caching = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.418494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.418565 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.418636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.418714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.418787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.418858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.418929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.419000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.419071 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.caching = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.419149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.419222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.419293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.419363 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.419434 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.419506 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.419585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.419659 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.419729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.419841 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.419924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420079 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420364 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.420953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.421025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.421102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.421175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.421246 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.421369 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.421452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.421513 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.421578 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.421649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.421720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.421800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.421905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.421905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422048 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422269 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.422893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.423020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.423098 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.423176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.423248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.423319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.423319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.423446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.423524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.423597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.423668 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.423739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.423739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.423992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.424606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.424682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.424754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.424825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.424896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.424967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.425047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.425120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.425192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.425263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.425263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.425390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.425469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.425541 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.425612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.425683 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.425753 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.425845 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.425933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.426008 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.426079 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.426149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.426219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.426289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.426367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.426440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.426511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.426640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.426717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.426784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.426870 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.427039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.427120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.427193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.427264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.427335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.427420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.427500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.427572 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.427644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.427795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.427876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.428019 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.428099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.428254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.428334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.428486 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.428573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.428718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.428797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.428943 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.429021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.429176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.429268 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.429415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.429494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.429651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.429730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.429915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.430017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.430146 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.430226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.430371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.430456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.430611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.430699 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.430843 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.430922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.431073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.431152 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.431302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.431389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.431531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.431610 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.431768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.431846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.432002 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.432089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.432234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.432312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.432469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.432565 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.432703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.432790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.432954 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.433048 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.433193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.433273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.433426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.433578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.433654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.433734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.433910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.433993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.434148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.434282 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.434363 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.434436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.434593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.434672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.434828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.434914 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] resource.caching = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.435070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.435148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.435310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.435387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.435539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.435626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.435774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.435852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.436015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] resource.caching = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.436094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.436242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.436328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.436485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.436564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.436717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.436795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.436949 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.437037 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.437179 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] role.caching = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.437259 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.437420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] role.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.437498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.437644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.437731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.437909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.437994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.438152 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.438226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.438393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] role.caching = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.443983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.443983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.444147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] role.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.444147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.444278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.444278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.444423 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.444498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.444570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.444641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.444711 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.444711 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.444845 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.444918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.444989 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.445060 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.445130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.445207 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.445280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.445350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.445420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.445490 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.445559 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.445644 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.445717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.445787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.445889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.445963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446748 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446818 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.446966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447038 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447109 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447179 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447396 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447468 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447754 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.447975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.448044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.448114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.448183 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.448261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] token.caching = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.448388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.448451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.448516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.448587 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.448667 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.448754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.448825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.448895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.448965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.449035 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.449112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] token.caching = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.449185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.449255 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.449324 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.449394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.449463 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.449541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.449612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.449682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.449753 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.449843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450096 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450218 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450359 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450506 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450579 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450738 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.450951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.connection = **** {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451625 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451697 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451837 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.connection = **** {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451915 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.451986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.452064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.452064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.452193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.452263 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.452332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.452402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.452564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.452640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.452710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.452781 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.452851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.452920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.452998 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.453070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.453140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.453210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.453280 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.453350 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.453427 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.453499 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.453579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.453650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.453720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.453790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81557) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 16 05:22:40.453963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.454200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.454445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.454675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.454917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.455182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 16 05:22:40.455372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81556) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 16 05:22:40.483210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.483434 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.483654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.483740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.483950 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.484181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.492671 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.493092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.520419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.520642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.521196 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.521431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.602518 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.602761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.650809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.651014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.651107 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.651245 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.810695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.810912 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.811130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.811346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.811561 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.811773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.821186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.821419 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.821643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.821891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.822115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.822334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.836564 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.836795 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.837018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.837236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.837453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.837666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.843037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.843145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.843145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.843145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.843145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.843145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.843145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.843526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.854186 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.854610 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.866806 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.867037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.867252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.867340 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.867471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.867554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.867686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.867771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.867896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.867983 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.868105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81557) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.868199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.868321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.868538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81556) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 16 05:22:40.884744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.884943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.885049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 16 05:22:40.885179 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 16 05:22:40.885263 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.885460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 16 05:22:40.885545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.885676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 16 05:22:40.885770 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.885977 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.886063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.886238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.886321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.886493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.886578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.886750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.896838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.897111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.897331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.897429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.897611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.897695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.897915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.898202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.904586 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.905152 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.907475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.907758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.908039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.908134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.908874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.909023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.909159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.909359 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.909505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.909657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.909783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.910030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.910307 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.910418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.910586 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.910672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.910858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.910939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.911163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.911439 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.911717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.911988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.933479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.934778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.934964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.935246 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.936252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.936537 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.939907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.940255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.940552 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.941475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.941475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.942067 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.946919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.947203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.948406 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.948694 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.952861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.953155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.953588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.953728 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.954013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.954289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.954475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.954574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.954774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.954857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.955066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.956001 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.956386 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.956670 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.956949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.957226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.971353 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.971711 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.973403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.973682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.975524 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.977572 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.978717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.980947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.981029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.981106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.983135 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.985321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.986234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.988580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:40.989532 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.989854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.990129 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.990416 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.990708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.990999 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.991844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.992147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.992437 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.992728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.993009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:40.993309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.006695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.006978 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.007241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.007503 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.009172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.009463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.009732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.010048 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.014378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.016165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.017128 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.018200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.021073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.021164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.021164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.021291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.021365 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.021598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.021911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.022192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.022575 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.024030 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.024316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.024607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.024892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.025285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.034269 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.034513 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.034804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.035068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.035441 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.035733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.036010 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.037169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.037457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.037741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.038051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.038434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.038724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.039005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.051445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.054338 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.054577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.057199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.057504 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.058731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.060218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.060418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.060507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.060870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.061884 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.063471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.063753 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.064151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.067972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.069416 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.071098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.072291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.072573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.072677 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.072955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.073228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.075500 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.075899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.076188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.076476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.081782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.082089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.082368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.084944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.085236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.085520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.089194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81557) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.089476 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.089748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.090040 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.090309 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.090577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.090844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.091132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.091395 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.091658 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.091930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.092192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.092527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81556) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 16 05:22:41.092616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.092693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.092768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81557) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.092964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.093242 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.093517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.093792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.094091 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.094378 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.094653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.094923 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.095217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.095484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.095778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.096169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81556) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 16 05:22:41.126659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557cce2569c0 pid: 81557 (default app) Sep 16 05:22:41.130621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fb580-803d-4ea2-a2a0-bf6913b77c90 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:41.130727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557cce2569c0 pid: 81556 (default app) Sep 16 05:22:41.130901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fb580-803d-4ea2-a2a0-bf6913b77c90 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:41.131169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fb580-803d-4ea2-a2a0-bf6913b77c90 None None] PATH_INFO: `/v3` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:41.135452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Fri Sep 16 05:22:41 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:22:48.904268 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fdeaa-62a2-42de-8454-33053c5c1da7 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:48.904789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fdeaa-62a2-42de-8454-33053c5c1da7 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:48.904789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fdeaa-62a2-42de-8454-33053c5c1da7 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:48.909534 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:22:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:22:48.915035 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3e0dc-2d96-49ae-9415-583671e26eeb None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:48.915497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3e0dc-2d96-49ae-9415-583671e26eeb None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:48.915596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3e0dc-2d96-49ae-9415-583671e26eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:48.994904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: INFO dbcounter [None req-4ef3e0dc-2d96-49ae-9415-583671e26eeb None None] Registered counter for database keystone Sep 16 05:22:49.014146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG oslo_db.sqlalchemy.engines [None req-4ef3e0dc-2d96-49ae-9415-583671e26eeb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81557) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 16 05:22:49.021688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writer thread running {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 16 05:22:49.204996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG passlib.handlers.bcrypt [None req-4ef3e0dc-2d96-49ae-9415-583671e26eeb None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81557) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 16 05:22:49.206652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG passlib.handlers.bcrypt [None req-4ef3e0dc-2d96-49ae-9415-583671e26eeb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81557) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 16 05:22:49.266765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-4ef3e0dc-2d96-49ae-9415-583671e26eeb None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:22:49.268933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4ef3e0dc-2d96-49ae-9415-583671e26eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:22:49.398274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:22:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 16 05:22:49.858417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fdeaa-62a2-42de-8454-33053c5c1da7 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:49.927865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: INFO dbcounter [None req-4e0fdeaa-62a2-42de-8454-33053c5c1da7 None None] Registered counter for database keystone Sep 16 05:22:49.949117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG oslo_db.sqlalchemy.engines [None req-4e0fdeaa-62a2-42de-8454-33053c5c1da7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81556) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 16 05:22:49.955685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writer thread running {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 16 05:22:50.076866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fdeaa-62a2-42de-8454-33053c5c1da7 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:22:50.081752 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc770e-34ff-4c7c-a7b5-d464a70193d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:22:50.082400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc770e-34ff-4c7c-a7b5-d464a70193d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:50.082677 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc770e-34ff-4c7c-a7b5-d464a70193d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:50.082945 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc770e-34ff-4c7c-a7b5-d464a70193d1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:50.091052 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc770e-34ff-4c7c-a7b5-d464a70193d1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:22:50.149227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: WARNING keystone.common.rbac_enforcer.enforcer [None req-83bc770e-34ff-4c7c-a7b5-d464a70193d1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 16 05:22:50.149610 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc770e-34ff-4c7c-a7b5-d464a70193d1 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:22:50.159986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Fri Sep 16 05:22:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:22:50.166088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3e0dc-2d96-49ae-9415-583671e26eeb None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:50.183066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3e0dc-2d96-49ae-9415-583671e26eeb None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:22:50.187485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719c6f1-7185-41b4-9419-a46180cc2f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:22:50.188600 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719c6f1-7185-41b4-9419-a46180cc2f42 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:50.188883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719c6f1-7185-41b4-9419-a46180cc2f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:50.189157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719c6f1-7185-41b4-9419-a46180cc2f42 admin admin] PATH_INFO: `/v3/projects/2c317c5acb304c5d983c73b9ed1b6802` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:50.191612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719c6f1-7185-41b4-9419-a46180cc2f42 admin admin] RBAC: Authorizing `identity:get_project(project_id=2c317c5acb304c5d983c73b9ed1b6802)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:22:50.244202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e719c6f1-7185-41b4-9419-a46180cc2f42 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 16 05:22:50.244540 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719c6f1-7185-41b4-9419-a46180cc2f42 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:22:50.249050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Fri Sep 16 05:22:50 2022] GET /identity/v3/projects/2c317c5acb304c5d983c73b9ed1b6802 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:22:52.443499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36f1ec-690f-4e54-a207-0bd5f765c75f None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:52.444021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36f1ec-690f-4e54-a207-0bd5f765c75f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:52.444105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36f1ec-690f-4e54-a207-0bd5f765c75f None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:52.448173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:22:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:22:52.453769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ade7f4-1ee5-44b2-a419-ead6265ca888 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:52.454172 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ade7f4-1ee5-44b2-a419-ead6265ca888 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:52.454337 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ade7f4-1ee5-44b2-a419-ead6265ca888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:52.474355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-96ade7f4-1ee5-44b2-a419-ead6265ca888 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:22:52.475835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-96ade7f4-1ee5-44b2-a419-ead6265ca888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:22:52.483436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:22:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 16 05:22:52.748468 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36f1ec-690f-4e54-a207-0bd5f765c75f None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:52.761130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36f1ec-690f-4e54-a207-0bd5f765c75f None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:22:52.765254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef0cf8-f8f6-420d-be3f-98663dbd2a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:22:52.766106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef0cf8-f8f6-420d-be3f-98663dbd2a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:52.766392 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef0cf8-f8f6-420d-be3f-98663dbd2a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:52.766657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef0cf8-f8f6-420d-be3f-98663dbd2a00 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:52.774324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef0cf8-f8f6-420d-be3f-98663dbd2a00 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:22:52.776056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef0cf8-f8f6-420d-be3f-98663dbd2a00 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:22:52.784629 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Fri Sep 16 05:22:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:22:52.790126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ade7f4-1ee5-44b2-a419-ead6265ca888 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:52.799854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ade7f4-1ee5-44b2-a419-ead6265ca888 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:22:52.804215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616ae88-de71-4008-aeb1-c3025c613db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:22:52.804836 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616ae88-de71-4008-aeb1-c3025c613db9 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:52.805112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616ae88-de71-4008-aeb1-c3025c613db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:52.805381 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616ae88-de71-4008-aeb1-c3025c613db9 admin admin] PATH_INFO: `/v3/users/2ebd28e621914177981b750fe6a7566b` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:52.806703 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616ae88-de71-4008-aeb1-c3025c613db9 admin admin] RBAC: Authorizing `identity:get_user(user_id=2ebd28e621914177981b750fe6a7566b)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:22:52.808812 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616ae88-de71-4008-aeb1-c3025c613db9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:22:52.810352 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Fri Sep 16 05:22:52 2022] GET /identity/v3/users/2ebd28e621914177981b750fe6a7566b => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:22:55.277971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f6b59-1d37-47b7-b320-1df36fb9be48 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:55.277971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f6b59-1d37-47b7-b320-1df36fb9be48 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:55.277971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f6b59-1d37-47b7-b320-1df36fb9be48 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:55.277971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:22:55.278719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1f487-d393-48e3-aa60-67232233cebe None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:55.278719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1f487-d393-48e3-aa60-67232233cebe None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:55.278719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1f487-d393-48e3-aa60-67232233cebe None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:55.278719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:22:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:22:55.279236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b1952-94a9-4328-9d82-652e73f6c27d None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:55.279236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b1952-94a9-4328-9d82-652e73f6c27d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:55.279236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b1952-94a9-4328-9d82-652e73f6c27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:55.279418 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3835eb4-102d-4397-8601-db6f29c81973 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:55.279418 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3835eb4-102d-4397-8601-db6f29c81973 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:55.279418 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3835eb4-102d-4397-8601-db6f29c81973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:55.280472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c3835eb4-102d-4397-8601-db6f29c81973 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:22:55.281635 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c3835eb4-102d-4397-8601-db6f29c81973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:22:55.298335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG passlib.handlers.bcrypt [None req-4e9b1952-94a9-4328-9d82-652e73f6c27d None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81556) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 16 05:22:55.299228 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG passlib.handlers.bcrypt [None req-4e9b1952-94a9-4328-9d82-652e73f6c27d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81556) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 16 05:22:55.348613 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-4e9b1952-94a9-4328-9d82-652e73f6c27d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:22:55.350258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4e9b1952-94a9-4328-9d82-652e73f6c27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:22:55.364497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 7/12] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:22:55.449420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 6/13] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:22:55.639998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3835eb4-102d-4397-8601-db6f29c81973 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:55.664341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:22:55.723813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b1952-94a9-4328-9d82-652e73f6c27d None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:55.744894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3835eb4-102d-4397-8601-db6f29c81973 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:22:55.779104 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfbf690-6f6f-4229-905b-de6e5e1331dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:22:55.779722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbf690-6f6f-4229-905b-de6e5e1331dd None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:55.780001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbf690-6f6f-4229-905b-de6e5e1331dd None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:55.780276 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbf690-6f6f-4229-905b-de6e5e1331dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:55.823233 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b1952-94a9-4328-9d82-652e73f6c27d None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:22:55.825488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:22:55.847160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfbf690-6f6f-4229-905b-de6e5e1331dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:22:55.848637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfbf690-6f6f-4229-905b-de6e5e1331dd None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:22:55.861981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd4931-9ecb-4e89-95b4-1dbca891f92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:22:55.863445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd4931-9ecb-4e89-95b4-1dbca891f92d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:55.863795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd4931-9ecb-4e89-95b4-1dbca891f92d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:55.864277 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd4931-9ecb-4e89-95b4-1dbca891f92d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:55.872940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:22:55.875978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd4931-9ecb-4e89-95b4-1dbca891f92d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-a2bd4931-9ecb-4e89-95b4-1dbca891f92d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:22:55.885010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:22:55.886860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:22:55.887802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:22:55.887802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:22:55.887802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:22:55.887802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:22:55.887802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:22:55.887802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:22:55.887802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:22:55.887802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:22:55.887802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:22:55.887802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:22:55.887802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:22:55.887802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:22:55.887802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:22:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:22:55.892277 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd4931-9ecb-4e89-95b4-1dbca891f92d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:55.949165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 8/15] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:22:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 05:22:55.954734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfbf690-6f6f-4229-905b-de6e5e1331dd None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:55.994179 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd4931-9ecb-4e89-95b4-1dbca891f92d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:22:56.027943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d7256-9173-48aa-8728-6a09d19e789e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:22:56.028543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d7256-9173-48aa-8728-6a09d19e789e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:56.028822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d7256-9173-48aa-8728-6a09d19e789e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:56.029093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d7256-9173-48aa-8728-6a09d19e789e None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:56.029754 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d7256-9173-48aa-8728-6a09d19e789e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:22:56.030082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d7256-9173-48aa-8728-6a09d19e789e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:22:56.031518 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d7256-9173-48aa-8728-6a09d19e789e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:22:56.043217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 8/16] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:22:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:22:56.049570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d7256-9173-48aa-8728-6a09d19e789e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:56.051013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfbf690-6f6f-4229-905b-de6e5e1331dd None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:22:56.073424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:22:56.076766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:22:56.084898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64299e-1485-4473-b1d6-5e1e4be58f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:22:56.085640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64299e-1485-4473-b1d6-5e1e4be58f39 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:56.101952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64299e-1485-4473-b1d6-5e1e4be58f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:56.101952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64299e-1485-4473-b1d6-5e1e4be58f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:56.101952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64299e-1485-4473-b1d6-5e1e4be58f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:22:56.101952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64299e-1485-4473-b1d6-5e1e4be58f39 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-ae64299e-1485-4473-b1d6-5e1e4be58f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:22:56.116604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:22:56.117502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:22:56.118341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:22:56.118341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:22:56.118341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:22:56.118341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 9/17] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:22:56.123447 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64299e-1485-4473-b1d6-5e1e4be58f39 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:56.148543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d7256-9173-48aa-8728-6a09d19e789e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:22:56.182150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e2b67-001c-4502-be72-33fcc8738033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:22:56.182772 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e2b67-001c-4502-be72-33fcc8738033 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:56.183041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e2b67-001c-4502-be72-33fcc8738033 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:56.183305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e2b67-001c-4502-be72-33fcc8738033 None admin] PATH_INFO: `/v3/users/2ebd28e621914177981b750fe6a7566b` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:56.184861 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e2b67-001c-4502-be72-33fcc8738033 None admin] RBAC: Authorizing `identity:get_user(user_id=2ebd28e621914177981b750fe6a7566b)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:22:56.186351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63e2b67-001c-4502-be72-33fcc8738033 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:22:56.192304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 9/18] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:22:56 2022] GET /identity/v3/users/2ebd28e621914177981b750fe6a7566b => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:22:56.193371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e2b67-001c-4502-be72-33fcc8738033 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:56.224960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64299e-1485-4473-b1d6-5e1e4be58f39 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:22:56.259001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe80c9-a684-4bc6-8831-1d746dc2049c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:22:56.259609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe80c9-a684-4bc6-8831-1d746dc2049c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:56.259889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe80c9-a684-4bc6-8831-1d746dc2049c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:56.260159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe80c9-a684-4bc6-8831-1d746dc2049c None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:56.260821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe80c9-a684-4bc6-8831-1d746dc2049c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:22:56.261104 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe80c9-a684-4bc6-8831-1d746dc2049c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:22:56.262517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe80c9-a684-4bc6-8831-1d746dc2049c None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:22:56.280467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 10/19] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:22:56.288194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:22:56.295141 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e2b67-001c-4502-be72-33fcc8738033 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:22:56.332929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ecb03-99c1-4fce-a4b6-32749095363b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:22:56.333589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ecb03-99c1-4fce-a4b6-32749095363b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:56.333897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ecb03-99c1-4fce-a4b6-32749095363b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:56.334169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ecb03-99c1-4fce-a4b6-32749095363b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:56.335378 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ecb03-99c1-4fce-a4b6-32749095363b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:22:56.336873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ecb03-99c1-4fce-a4b6-32749095363b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:22:56.338550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 10/20] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:22:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:22:56.344818 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe80c9-a684-4bc6-8831-1d746dc2049c None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:56.382581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:22:56.451824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe80c9-a684-4bc6-8831-1d746dc2049c None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:22:56.488702 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3648f-5b5e-4517-80cc-cf96d677d6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:22:56.489309 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3648f-5b5e-4517-80cc-cf96d677d6d9 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:56.489588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3648f-5b5e-4517-80cc-cf96d677d6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:56.489885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3648f-5b5e-4517-80cc-cf96d677d6d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:56.490612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3648f-5b5e-4517-80cc-cf96d677d6d9 None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.domain.id=default, user.id=2ebd28e621914177981b750fe6a7566b) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:22:56.490918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3648f-5b5e-4517-80cc-cf96d677d6d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:22:56.492368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d3648f-5b5e-4517-80cc-cf96d677d6d9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:22:56.499165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 11/21] 10.222.0.49 () {56 vars in 1322 bytes} [Fri Sep 16 05:22:56 2022] GET /identity/v3/role_assignments?user.id=2ebd28e621914177981b750fe6a7566b&scope.domain.id=default&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:22:58.992450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ba8f4-ff01-4ffa-accc-eadcc5036688 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:58.993438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ba8f4-ff01-4ffa-accc-eadcc5036688 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:58.993940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ba8f4-ff01-4ffa-accc-eadcc5036688 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:58.995824 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 11/22] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:22:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:22:59.000861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5e528-df9c-4267-8fec-ee8339cbc3ef None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:59.001735 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5e528-df9c-4267-8fec-ee8339cbc3ef None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:59.002241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5e528-df9c-4267-8fec-ee8339cbc3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:59.026811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-88a5e528-df9c-4267-8fec-ee8339cbc3ef None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:22:59.028380 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-88a5e528-df9c-4267-8fec-ee8339cbc3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:22:59.121416 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 12/23] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:22:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:22:59.238479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a9c13-931e-42bc-92fc-95fe5e51d169 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:59.239086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a9c13-931e-42bc-92fc-95fe5e51d169 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:59.239552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a9c13-931e-42bc-92fc-95fe5e51d169 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:59.241088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 12/24] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:22:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:22:59.246526 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105918c-dacc-4c8b-8bb2-f344de61090c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:59.247054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105918c-dacc-4c8b-8bb2-f344de61090c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:59.247521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105918c-dacc-4c8b-8bb2-f344de61090c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:59.274796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a105918c-dacc-4c8b-8bb2-f344de61090c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:22:59.276348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a105918c-dacc-4c8b-8bb2-f344de61090c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:22:59.308558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:22:59.362668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 13/25] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:22:59.364875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6b624-f0da-4f58-8c62-067bd2e716a8 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:59.365161 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6b624-f0da-4f58-8c62-067bd2e716a8 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:59.365435 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6b624-f0da-4f58-8c62-067bd2e716a8 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:59.367020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 13/26] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:22:59.371894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6f85a-0cee-4eb6-a472-ba24befd2776 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:59.372183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6f85a-0cee-4eb6-a472-ba24befd2776 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:59.372468 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6f85a-0cee-4eb6-a472-ba24befd2776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:59.394465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-bde6f85a-0cee-4eb6-a472-ba24befd2776 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:22:59.395674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-bde6f85a-0cee-4eb6-a472-ba24befd2776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:22:59.491269 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 14/27] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:22:59.617405 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6b624-f0da-4f58-8c62-067bd2e716a8 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:59.683382 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80f439-fa4b-4691-bcd0-16908bc084b5 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:59.683714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80f439-fa4b-4691-bcd0-16908bc084b5 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:59.683927 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c80f439-fa4b-4691-bcd0-16908bc084b5 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:59.685494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 15/28] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:22:59.690457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164354e4-f7a5-480f-ab21-1bc4e7b0574c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:59.690733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164354e4-f7a5-480f-ab21-1bc4e7b0574c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:59.691005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164354e4-f7a5-480f-ab21-1bc4e7b0574c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:59.721654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-164354e4-f7a5-480f-ab21-1bc4e7b0574c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:22:59.722851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-164354e4-f7a5-480f-ab21-1bc4e7b0574c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:22:59.842379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6b624-f0da-4f58-8c62-067bd2e716a8 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:22:59.877934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 16/29] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:22:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:22:59.878847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164354e4-f7a5-480f-ab21-1bc4e7b0574c None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:59.888443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a5cd-e404-4758-ae2d-079dfcffddb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:22:59.889393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a5cd-e404-4758-ae2d-079dfcffddb6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:22:59.889888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a5cd-e404-4758-ae2d-079dfcffddb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:22:59.890372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a5cd-e404-4758-ae2d-079dfcffddb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:22:59.915318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82a5cd-e404-4758-ae2d-079dfcffddb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:22:59.917265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82a5cd-e404-4758-ae2d-079dfcffddb6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-fe82a5cd-e404-4758-ae2d-079dfcffddb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:22:59.935910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:22:59.936845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:22:59.937634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:22:59.937634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:22:59.937634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:22:59.939281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:22:59.940942 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 14/30] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:22:59.942168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a5cd-e404-4758-ae2d-079dfcffddb6 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:22:59.961679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:00.060545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a5cd-e404-4758-ae2d-079dfcffddb6 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:00.087718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164354e4-f7a5-480f-ab21-1bc4e7b0574c None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:00.102759 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5857fba-8fe1-4128-bf2c-9fb641dd8461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:00.104042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5857fba-8fe1-4128-bf2c-9fb641dd8461 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:00.104745 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5857fba-8fe1-4128-bf2c-9fb641dd8461 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:00.105181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5857fba-8fe1-4128-bf2c-9fb641dd8461 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:00.106142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5857fba-8fe1-4128-bf2c-9fb641dd8461 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:00.107802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5857fba-8fe1-4128-bf2c-9fb641dd8461 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:00.128361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d976fc-7c20-4e93-a9cc-4f2ca81b4ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:00.129101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d976fc-7c20-4e93-a9cc-4f2ca81b4ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:00.129444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d976fc-7c20-4e93-a9cc-4f2ca81b4ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:00.129854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d976fc-7c20-4e93-a9cc-4f2ca81b4ca9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:00.142445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d976fc-7c20-4e93-a9cc-4f2ca81b4ca9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-d3d976fc-7c20-4e93-a9cc-4f2ca81b4ca9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:00.150437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:00.156775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 17/31] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:22:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:00.157595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d976fc-7c20-4e93-a9cc-4f2ca81b4ca9 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:00.263101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:00.288196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d976fc-7c20-4e93-a9cc-4f2ca81b4ca9 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:00.324079 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b8876-710a-465e-bbe7-727e41096d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:00.325133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b8876-710a-465e-bbe7-727e41096d11 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:00.325675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b8876-710a-465e-bbe7-727e41096d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:00.326178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b8876-710a-465e-bbe7-727e41096d11 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:00.327008 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b8876-710a-465e-bbe7-727e41096d11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:00.327493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b8876-710a-465e-bbe7-727e41096d11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:00.329101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b8876-710a-465e-bbe7-727e41096d11 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:00.338979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 15/32] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Sep 16 05:22:59 2022] POST /identity/v3/roles => generated 219 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:00.338979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5857fba-8fe1-4128-bf2c-9fb641dd8461 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:00.360087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 18/33] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:00.361672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b8876-710a-465e-bbe7-727e41096d11 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:00.520472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:00.523683 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b8876-710a-465e-bbe7-727e41096d11 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:00.572193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5857fba-8fe1-4128-bf2c-9fb641dd8461 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:00.586373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42933e-f00e-4517-adcb-e870ec991144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:00.587006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42933e-f00e-4517-adcb-e870ec991144 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:00.587280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42933e-f00e-4517-adcb-e870ec991144 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:00.588241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42933e-f00e-4517-adcb-e870ec991144 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:00.589018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42933e-f00e-4517-adcb-e870ec991144 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:00.590514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42933e-f00e-4517-adcb-e870ec991144 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:00.629499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd2bca-cc99-4be8-9e30-7fa57d359d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:00.630201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd2bca-cc99-4be8-9e30-7fa57d359d42 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:00.630478 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd2bca-cc99-4be8-9e30-7fa57d359d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:00.630785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd2bca-cc99-4be8-9e30-7fa57d359d42 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:00.631426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd2bca-cc99-4be8-9e30-7fa57d359d42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:00.631727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd2bca-cc99-4be8-9e30-7fa57d359d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:00.633229 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbd2bca-cc99-4be8-9e30-7fa57d359d42 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:00.645612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 16/34] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:00.646859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd2bca-cc99-4be8-9e30-7fa57d359d42 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:00.737972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:00.754452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd2bca-cc99-4be8-9e30-7fa57d359d42 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:00.788360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e28e8d-1971-4ff3-8869-632feb3b67e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:00.788969 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e28e8d-1971-4ff3-8869-632feb3b67e2 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:00.789245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e28e8d-1971-4ff3-8869-632feb3b67e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:00.789515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e28e8d-1971-4ff3-8869-632feb3b67e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:00.790195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e28e8d-1971-4ff3-8869-632feb3b67e2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:00.791621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e28e8d-1971-4ff3-8869-632feb3b67e2 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:00.797354 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:00.848969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 19/35] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Sep 16 05:23:00 2022] POST /identity/v3/roles => generated 225 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:00.850006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42933e-f00e-4517-adcb-e870ec991144 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:00.909499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 17/36] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:23:00 2022] POST /identity/v3/projects => generated 301 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:00.911163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad77b4b-7d52-4321-be31-50f99aa26cc7 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:00.911501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad77b4b-7d52-4321-be31-50f99aa26cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:00.911863 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad77b4b-7d52-4321-be31-50f99aa26cc7 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:00.913594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 18/37] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:00.914815 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21951c74-b07b-41bc-8438-1b1d56cb9146 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:00.915174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21951c74-b07b-41bc-8438-1b1d56cb9146 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:00.915521 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21951c74-b07b-41bc-8438-1b1d56cb9146 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:00.921526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 19/38] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:00.921526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d66ae1-a2e3-4e50-92a3-f40f8c3aa17c None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:00.921526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d66ae1-a2e3-4e50-92a3-f40f8c3aa17c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:00.921526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d66ae1-a2e3-4e50-92a3-f40f8c3aa17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:00.946848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-d0d66ae1-a2e3-4e50-92a3-f40f8c3aa17c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:00.948440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d0d66ae1-a2e3-4e50-92a3-f40f8c3aa17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:01.008113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42933e-f00e-4517-adcb-e870ec991144 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:01.036783 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 20/39] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:01.038497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606b319-fe16-4cd7-99f7-813e136a84a7 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:01.038933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606b319-fe16-4cd7-99f7-813e136a84a7 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:01.039682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606b319-fe16-4cd7-99f7-813e136a84a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:01.057141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a815bf-e740-404a-9c99-33bf965de337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:01.058083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a815bf-e740-404a-9c99-33bf965de337 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:01.058510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a815bf-e740-404a-9c99-33bf965de337 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:01.058950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a815bf-e740-404a-9c99-33bf965de337 None admin] PATH_INFO: `/v3/users/2ebd28e621914177981b750fe6a7566b` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:01.060553 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a815bf-e740-404a-9c99-33bf965de337 None admin] RBAC: Authorizing `identity:get_user(user_id=2ebd28e621914177981b750fe6a7566b)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:01.061004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-2606b319-fe16-4cd7-99f7-813e136a84a7 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:01.062851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a815bf-e740-404a-9c99-33bf965de337 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:01.063151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2606b319-fe16-4cd7-99f7-813e136a84a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:01.065215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 20/40] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:00 2022] GET /identity/v3/users/2ebd28e621914177981b750fe6a7566b => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:01.073489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a815bf-e740-404a-9c99-33bf965de337 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:01.166535 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 21/41] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:01.262283 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:01.297881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a815bf-e740-404a-9c99-33bf965de337 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:01.323636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606b319-fe16-4cd7-99f7-813e136a84a7 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:01.333545 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c61e19-af6d-432f-b2e2-bba1ca21ef91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:01.334570 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c61e19-af6d-432f-b2e2-bba1ca21ef91 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:01.334878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c61e19-af6d-432f-b2e2-bba1ca21ef91 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:01.335156 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c61e19-af6d-432f-b2e2-bba1ca21ef91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:01.336688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c61e19-af6d-432f-b2e2-bba1ca21ef91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:01.338176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c61e19-af6d-432f-b2e2-bba1ca21ef91 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:01.344383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 21/42] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:01.345200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c61e19-af6d-432f-b2e2-bba1ca21ef91 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:01.379900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:01.433466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606b319-fe16-4cd7-99f7-813e136a84a7 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:01.456907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c61e19-af6d-432f-b2e2-bba1ca21ef91 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:01.471952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2dd82-4756-4ebe-9eb4-67734b356e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:01.472553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2dd82-4756-4ebe-9eb4-67734b356e70 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:01.472836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2dd82-4756-4ebe-9eb4-67734b356e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:01.473105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2dd82-4756-4ebe-9eb4-67734b356e70 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:01.473778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2dd82-4756-4ebe-9eb4-67734b356e70 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:01.475181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2dd82-4756-4ebe-9eb4-67734b356e70 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:01.490165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0d301-d098-4490-b176-69cf79fa4934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:01.490827 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0d301-d098-4490-b176-69cf79fa4934 None admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:01.491108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0d301-d098-4490-b176-69cf79fa4934 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:01.491384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0d301-d098-4490-b176-69cf79fa4934 None admin] PATH_INFO: `/v3/domains/default/users/2ebd28e621914177981b750fe6a7566b/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:01.493370 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0d301-d098-4490-b176-69cf79fa4934 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2ebd28e621914177981b750fe6a7566b, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:01.494843 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0d301-d098-4490-b176-69cf79fa4934 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:01.497036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:01.571300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 22/43] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Sep 16 05:23:01 2022] POST /identity/v3/roles => generated 223 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:23:01.572139 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2dd82-4756-4ebe-9eb4-67734b356e70 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:01.589975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 22/44] 10.222.0.49 () {58 vars in 1361 bytes} [Fri Sep 16 05:23:01 2022] PUT /identity/v3/domains/default/users/2ebd28e621914177981b750fe6a7566b/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:23:01.702274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2dd82-4756-4ebe-9eb4-67734b356e70 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:01.713546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:01.743150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ff561-5fd1-448e-9bc1-8268b36d5aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:01.743823 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ff561-5fd1-448e-9bc1-8268b36d5aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:01.744103 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ff561-5fd1-448e-9bc1-8268b36d5aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:01.744375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ff561-5fd1-448e-9bc1-8268b36d5aa4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:01.745623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ff561-5fd1-448e-9bc1-8268b36d5aa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:01.747153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ff561-5fd1-448e-9bc1-8268b36d5aa4 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:01.749097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 23/45] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:01.755468 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0d301-d098-4490-b176-69cf79fa4934 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:01.860571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0d301-d098-4490-b176-69cf79fa4934 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:01.893917 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac81770-a9a1-45b5-976f-ef004d535b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:01.894520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac81770-a9a1-45b5-976f-ef004d535b1b None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:01.894794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac81770-a9a1-45b5-976f-ef004d535b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:01.895063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac81770-a9a1-45b5-976f-ef004d535b1b None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:01.895712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac81770-a9a1-45b5-976f-ef004d535b1b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:01.897157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac81770-a9a1-45b5-976f-ef004d535b1b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:01.973430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 23/46] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:23:01 2022] POST /identity/v3/projects => generated 312 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:03.758955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10095bf9-574c-471c-b040-6c580cda0b53 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:03.759491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10095bf9-574c-471c-b040-6c580cda0b53 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:03.759491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10095bf9-574c-471c-b040-6c580cda0b53 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:03.760814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 24/47] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:03.766323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcf442-bc5a-47a7-a388-1ba64377facb None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:03.766813 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcf442-bc5a-47a7-a388-1ba64377facb None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:03.767143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcf442-bc5a-47a7-a388-1ba64377facb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:03.785708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-d2bcf442-bc5a-47a7-a388-1ba64377facb None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:03.787218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d2bcf442-bc5a-47a7-a388-1ba64377facb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:03.828050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:03.851409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 24/48] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:04.119836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10095bf9-574c-471c-b040-6c580cda0b53 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:04.145465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcfb69-3947-4f2d-8d2e-4861731b9c27 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:04.145748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcfb69-3947-4f2d-8d2e-4861731b9c27 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:04.146049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcfb69-3947-4f2d-8d2e-4861731b9c27 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:04.147377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 25/49] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:04.152347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06f24a-05cb-420c-8e15-e568225b1a3d None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:04.152633 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06f24a-05cb-420c-8e15-e568225b1a3d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:04.152911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06f24a-05cb-420c-8e15-e568225b1a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:04.170478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-7b06f24a-05cb-420c-8e15-e568225b1a3d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:04.171657 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7b06f24a-05cb-420c-8e15-e568225b1a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:04.225851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10095bf9-574c-471c-b040-6c580cda0b53 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:04.238290 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 26/50] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:04.259080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb78b24-9c99-4e2b-b49c-af72a9bb6135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:04.259698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb78b24-9c99-4e2b-b49c-af72a9bb6135 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:04.259967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb78b24-9c99-4e2b-b49c-af72a9bb6135 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:04.260229 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb78b24-9c99-4e2b-b49c-af72a9bb6135 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:04.270249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb78b24-9c99-4e2b-b49c-af72a9bb6135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-2bb78b24-9c99-4e2b-b49c-af72a9bb6135 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:04.275442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:04.276328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:04.277254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:04.277254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:04.277254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:04.277254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:04.277254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:04.277254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:04.277254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:04.277254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:04.277254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:04.277254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:04.277254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:04.277254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:04.277254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 25/51] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:23:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:04.282223 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06f24a-05cb-420c-8e15-e568225b1a3d None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:04.379497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06f24a-05cb-420c-8e15-e568225b1a3d None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:04.386779 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:04.412846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173cf1c-a008-47c2-96ea-130cface211e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:04.413452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173cf1c-a008-47c2-96ea-130cface211e None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:04.413732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173cf1c-a008-47c2-96ea-130cface211e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:04.414033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173cf1c-a008-47c2-96ea-130cface211e None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:04.414700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173cf1c-a008-47c2-96ea-130cface211e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:04.415000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173cf1c-a008-47c2-96ea-130cface211e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:04.416370 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173cf1c-a008-47c2-96ea-130cface211e None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:04.428327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 27/52] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:04.434071 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb78b24-9c99-4e2b-b49c-af72a9bb6135 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:04.467182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:04.511787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173cf1c-a008-47c2-96ea-130cface211e None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:04.535467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb78b24-9c99-4e2b-b49c-af72a9bb6135 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:04.569958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9b119-15e2-4fe4-a1c3-9d9ec3439aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:04.570585 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9b119-15e2-4fe4-a1c3-9d9ec3439aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:04.570864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9b119-15e2-4fe4-a1c3-9d9ec3439aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:04.571176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9b119-15e2-4fe4-a1c3-9d9ec3439aa0 None admin] PATH_INFO: `/v3/users/2ebd28e621914177981b750fe6a7566b` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:04.572293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9b119-15e2-4fe4-a1c3-9d9ec3439aa0 None admin] RBAC: Authorizing `identity:get_user(user_id=2ebd28e621914177981b750fe6a7566b)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:04.573732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9b119-15e2-4fe4-a1c3-9d9ec3439aa0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:04.575329 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 26/53] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:04 2022] GET /identity/v3/users/2ebd28e621914177981b750fe6a7566b => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:04.580753 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9b119-15e2-4fe4-a1c3-9d9ec3439aa0 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:04.616618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173cf1c-a008-47c2-96ea-130cface211e None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:04.651612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1e975-a7de-4110-9b99-a24c4faa36c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:04.652171 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1e975-a7de-4110-9b99-a24c4faa36c4 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:04.652449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1e975-a7de-4110-9b99-a24c4faa36c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:04.652722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1e975-a7de-4110-9b99-a24c4faa36c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:04.653863 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1e975-a7de-4110-9b99-a24c4faa36c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:04.655303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1e975-a7de-4110-9b99-a24c4faa36c4 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:04.657023 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 28/54] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:04.663024 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1e975-a7de-4110-9b99-a24c4faa36c4 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:04.682636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9b119-15e2-4fe4-a1c3-9d9ec3439aa0 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:04.684993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:04.697497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:04.716770 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a96b4-aac5-4251-a3fd-0456189fa5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:04.717391 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a96b4-aac5-4251-a3fd-0456189fa5ce None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:04.717668 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a96b4-aac5-4251-a3fd-0456189fa5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:04.717958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a96b4-aac5-4251-a3fd-0456189fa5ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:04.719294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a96b4-aac5-4251-a3fd-0456189fa5ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:04.723775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a96b4-aac5-4251-a3fd-0456189fa5ce None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:04.723775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 27/55] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:04.729380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a96b4-aac5-4251-a3fd-0456189fa5ce None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:04.773794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1e975-a7de-4110-9b99-a24c4faa36c4 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:04.824106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62cbc6-772f-41a2-92ff-23bcab2fb055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:04.824851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62cbc6-772f-41a2-92ff-23bcab2fb055 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:04.825275 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62cbc6-772f-41a2-92ff-23bcab2fb055 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:04.825660 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62cbc6-772f-41a2-92ff-23bcab2fb055 None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:04.826467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62cbc6-772f-41a2-92ff-23bcab2fb055 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:04.828095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff62cbc6-772f-41a2-92ff-23bcab2fb055 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:04.858065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a96b4-aac5-4251-a3fd-0456189fa5ce None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:04.966902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 29/56] 10.222.0.49 () {60 vars in 1140 bytes} [Fri Sep 16 05:23:04 2022] POST /identity/v3/projects => generated 298 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:23:04.999391 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5ba46-0fed-4ec2-89f8-3d4ea9a38c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:05.000012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5ba46-0fed-4ec2-89f8-3d4ea9a38c91 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:05.000288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5ba46-0fed-4ec2-89f8-3d4ea9a38c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:05.000554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5ba46-0fed-4ec2-89f8-3d4ea9a38c91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:05.001300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5ba46-0fed-4ec2-89f8-3d4ea9a38c91 None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.domain.id=default, user.id=2ebd28e621914177981b750fe6a7566b) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:05.001690 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5ba46-0fed-4ec2-89f8-3d4ea9a38c91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:05.003671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5ba46-0fed-4ec2-89f8-3d4ea9a38c91 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:05.013189 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 28/57] 10.222.0.49 () {56 vars in 1322 bytes} [Fri Sep 16 05:23:04 2022] GET /identity/v3/role_assignments?user.id=2ebd28e621914177981b750fe6a7566b&scope.domain.id=default&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 530 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:07.125850 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f360b-efb4-475b-8533-df0622882d71 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:07.126356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f360b-efb4-475b-8533-df0622882d71 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:07.126500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f360b-efb4-475b-8533-df0622882d71 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:07.127874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 30/58] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:07.133016 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61233365-64bd-4d00-9a34-70a1db4e20b3 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:07.133375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61233365-64bd-4d00-9a34-70a1db4e20b3 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:07.133553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61233365-64bd-4d00-9a34-70a1db4e20b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:07.151840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-61233365-64bd-4d00-9a34-70a1db4e20b3 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:07.153021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-61233365-64bd-4d00-9a34-70a1db4e20b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:07.181287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:07.216687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 29/59] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:07.482700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f360b-efb4-475b-8533-df0622882d71 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:07.573750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:07.586486 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f360b-efb4-475b-8533-df0622882d71 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:07.624691 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e4cc3-2fbf-416d-86a0-6df63df3de5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:07.625315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e4cc3-2fbf-416d-86a0-6df63df3de5f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:07.625590 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e4cc3-2fbf-416d-86a0-6df63df3de5f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:07.625879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e4cc3-2fbf-416d-86a0-6df63df3de5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:07.627144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e4cc3-2fbf-416d-86a0-6df63df3de5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:07.628586 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e4cc3-2fbf-416d-86a0-6df63df3de5f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:07.630162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 31/60] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:07.635654 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61233365-64bd-4d00-9a34-70a1db4e20b3 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:07.732468 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61233365-64bd-4d00-9a34-70a1db4e20b3 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:07.765549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cf217-da5f-4657-be35-d3770b02d51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:07.766377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cf217-da5f-4657-be35-d3770b02d51b None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:07.766652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cf217-da5f-4657-be35-d3770b02d51b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:07.766921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cf217-da5f-4657-be35-d3770b02d51b None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:07.768202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cf217-da5f-4657-be35-d3770b02d51b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:07.769632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704cf217-da5f-4657-be35-d3770b02d51b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:07.855768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 30/61] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:23:07 2022] POST /identity/v3/users => generated 275 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:11.275231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c8e2a-26e5-48e4-87a7-522d4315a154 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:11.276224 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c8e2a-26e5-48e4-87a7-522d4315a154 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:11.276760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c8e2a-26e5-48e4-87a7-522d4315a154 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:11.278329 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 32/62] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:11.284403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3f685-1339-4d92-888e-a4f83b78394d None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:11.285203 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3f685-1339-4d92-888e-a4f83b78394d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:11.285765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3f685-1339-4d92-888e-a4f83b78394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:11.306515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-3bc3f685-1339-4d92-888e-a4f83b78394d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:11.307993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3bc3f685-1339-4d92-888e-a4f83b78394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:11.328082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:11.363180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648561f-fbe1-4822-9795-7ecd1023e02b None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:11.363506 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648561f-fbe1-4822-9795-7ecd1023e02b None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:11.363716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648561f-fbe1-4822-9795-7ecd1023e02b None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:11.365246 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 33/63] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:11.370709 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0433d8-b272-4363-8beb-fa487726d18c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:11.371233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0433d8-b272-4363-8beb-fa487726d18c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:11.371724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0433d8-b272-4363-8beb-fa487726d18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:11.396375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-1f0433d8-b272-4363-8beb-fa487726d18c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:11.397942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-1f0433d8-b272-4363-8beb-fa487726d18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:11.413624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 31/64] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:11.516732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 34/65] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:11.628163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93ce5e-9dfe-4082-9b5c-9e394985f8a6 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:11.628477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93ce5e-9dfe-4082-9b5c-9e394985f8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:11.628709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93ce5e-9dfe-4082-9b5c-9e394985f8a6 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:11.630076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 32/66] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:23:11.633888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9c95e-ba4e-4fa0-8a5e-1a7ff86d807a None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:11.634440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9c95e-ba4e-4fa0-8a5e-1a7ff86d807a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:11.634902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9c95e-ba4e-4fa0-8a5e-1a7ff86d807a None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:11.635786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ad7e6-9f7a-4af4-ada0-c988bf171e25 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:11.636257 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ad7e6-9f7a-4af4-ada0-c988bf171e25 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:11.636442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 35/67] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:11.636861 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ad7e6-9f7a-4af4-ada0-c988bf171e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:11.641969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c493120-c13f-4568-a69e-cad177db59fe None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:11.642585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c493120-c13f-4568-a69e-cad177db59fe None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:11.643069 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c493120-c13f-4568-a69e-cad177db59fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:11.667787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-6d8ad7e6-9f7a-4af4-ada0-c988bf171e25 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:11.669310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6d8ad7e6-9f7a-4af4-ada0-c988bf171e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:11.677105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-4c493120-c13f-4568-a69e-cad177db59fe None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:11.681363 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4c493120-c13f-4568-a69e-cad177db59fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:11.774913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 33/68] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:11.785260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ad7e6-9f7a-4af4-ada0-c988bf171e25 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:11.824149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 36/69] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:11.917051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b5552-10ff-4983-8934-19bad8a0fb50 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:11.917678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b5552-10ff-4983-8934-19bad8a0fb50 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:11.918222 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b5552-10ff-4983-8934-19bad8a0fb50 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:11.919753 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 37/70] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:11.923008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:11.925413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b5552-10ff-4983-8934-19bad8a0fb50 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:11.947029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ad7e6-9f7a-4af4-ada0-c988bf171e25 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:11.951420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:11.993611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3442e02-6a3c-46ad-ae8e-6e69a07eb753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:11.994283 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3442e02-6a3c-46ad-ae8e-6e69a07eb753 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:11.994562 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3442e02-6a3c-46ad-ae8e-6e69a07eb753 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:11.994833 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3442e02-6a3c-46ad-ae8e-6e69a07eb753 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:12.006436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3442e02-6a3c-46ad-ae8e-6e69a07eb753 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-e3442e02-6a3c-46ad-ae8e-6e69a07eb753 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:12.011885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:12.012734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:12.013495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:12.013495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:12.013495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:12.013495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:12.013495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:12.013495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:12.013495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:12.013495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:12.013495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:12.013495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:12.013495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:12.013495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:12.013495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 34/71] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:23:12.016658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5d09c-50cf-4b52-ac79-34e6bc086ec3 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:12.016966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5d09c-50cf-4b52-ac79-34e6bc086ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:12.017251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5d09c-50cf-4b52-ac79-34e6bc086ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:12.037759 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-04e5d09c-50cf-4b52-ac79-34e6bc086ec3 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:12.038983 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-04e5d09c-50cf-4b52-ac79-34e6bc086ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:12.062141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b5552-10ff-4983-8934-19bad8a0fb50 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:12.098724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e161525-d799-4160-a621-d587ff25c3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:12.099355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e161525-d799-4160-a621-d587ff25c3cf None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:12.099638 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e161525-d799-4160-a621-d587ff25c3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:12.099908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e161525-d799-4160-a621-d587ff25c3cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:12.110800 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e161525-d799-4160-a621-d587ff25c3cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:12.114713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 35/72] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:12.115379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5d09c-50cf-4b52-ac79-34e6bc086ec3 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-5e161525-d799-4160-a621-d587ff25c3cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:12.116745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:12.117589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 38/73] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:23:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:12.119049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e161525-d799-4160-a621-d587ff25c3cf None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:12.195159 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:12.208054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:12.228703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5d09c-50cf-4b52-ac79-34e6bc086ec3 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:12.231897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e161525-d799-4160-a621-d587ff25c3cf None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:12.266526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac9ce8-517e-4903-a145-c7355edbba47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:12.267181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac9ce8-517e-4903-a145-c7355edbba47 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:12.267460 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac9ce8-517e-4903-a145-c7355edbba47 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:12.267738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac9ce8-517e-4903-a145-c7355edbba47 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:12.268395 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac9ce8-517e-4903-a145-c7355edbba47 None admin] RBAC: Adding query filter params (name=member) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:12.268560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3361d6-394f-481e-8e02-3178f3eb1dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:12.268739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac9ce8-517e-4903-a145-c7355edbba47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:12.269193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3361d6-394f-481e-8e02-3178f3eb1dea None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:12.269475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3361d6-394f-481e-8e02-3178f3eb1dea None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:12.269752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3361d6-394f-481e-8e02-3178f3eb1dea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:12.270106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac9ce8-517e-4903-a145-c7355edbba47 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:12.283133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 36/74] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:12.283133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac9ce8-517e-4903-a145-c7355edbba47 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:12.284504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3361d6-394f-481e-8e02-3178f3eb1dea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-2f3361d6-394f-481e-8e02-3178f3eb1dea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:12.289494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:12.290401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:12.291169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:12.291169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:12.291169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:12.291169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:12.291169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:12.291169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:12.291169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:12.291169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:12.291169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:12.291169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:12.291169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:12.291169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:12.291169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 39/75] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:23:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:12.291798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3361d6-394f-481e-8e02-3178f3eb1dea None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:12.390510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac9ce8-517e-4903-a145-c7355edbba47 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:12.400482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3361d6-394f-481e-8e02-3178f3eb1dea None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:12.424105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad22ae-1b58-40cc-a879-3a1e4532da7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:12.424727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad22ae-1b58-40cc-a879-3a1e4532da7f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:12.425004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad22ae-1b58-40cc-a879-3a1e4532da7f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:12.425270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ad22ae-1b58-40cc-a879-3a1e4532da7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:12.427130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:12.434172 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a89916-ef56-4cf5-9f7e-29b75a4ead9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:12.434781 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a89916-ef56-4cf5-9f7e-29b75a4ead9c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:12.435059 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a89916-ef56-4cf5-9f7e-29b75a4ead9c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:12.435334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a89916-ef56-4cf5-9f7e-29b75a4ead9c None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:12.435960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ad22ae-1b58-40cc-a879-3a1e4532da7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:12.436048 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a89916-ef56-4cf5-9f7e-29b75a4ead9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:12.436291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a89916-ef56-4cf5-9f7e-29b75a4ead9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:12.437689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a89916-ef56-4cf5-9f7e-29b75a4ead9c None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:12.439976 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-56ad22ae-1b58-40cc-a879-3a1e4532da7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:12.440935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:12.441904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:12.442672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:12.442672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:12.442672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:12.442672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:12.442672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:12.442672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:12.442672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:12.442672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:12.442672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:12.442672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:12.442672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:12.442672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:12.442672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 37/76] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:12.443314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad22ae-1b58-40cc-a879-3a1e4532da7f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:12.451543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 40/77] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:12.451543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a89916-ef56-4cf5-9f7e-29b75a4ead9c None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:12.544806 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ad22ae-1b58-40cc-a879-3a1e4532da7f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:12.547897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a89916-ef56-4cf5-9f7e-29b75a4ead9c None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:12.578182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7e278-0265-4251-82cc-5e4ddf4c6276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:12.578797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7e278-0265-4251-82cc-5e4ddf4c6276 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:12.579074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7e278-0265-4251-82cc-5e4ddf4c6276 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:12.579347 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7e278-0265-4251-82cc-5e4ddf4c6276 None admin] PATH_INFO: `/v3/users/fd426537f3dc4593941437dc64e7c961` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:12.581157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d60e6-216a-4788-9fff-2ecbc2667e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:12.581766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d60e6-216a-4788-9fff-2ecbc2667e31 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:12.582079 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d60e6-216a-4788-9fff-2ecbc2667e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:12.582363 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d60e6-216a-4788-9fff-2ecbc2667e31 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:12.583131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d60e6-216a-4788-9fff-2ecbc2667e31 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:12.583444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d60e6-216a-4788-9fff-2ecbc2667e31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:12.584833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d60e6-216a-4788-9fff-2ecbc2667e31 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:12.592619 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 41/78] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:23:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:12.593411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d60e6-216a-4788-9fff-2ecbc2667e31 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:12.594905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7e278-0265-4251-82cc-5e4ddf4c6276 None admin] RBAC: Authorizing `identity:get_user(user_id=fd426537f3dc4593941437dc64e7c961)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:12.596979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7e278-0265-4251-82cc-5e4ddf4c6276 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:12.598620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 38/79] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:12 2022] GET /identity/v3/users/fd426537f3dc4593941437dc64e7c961 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:12.599369 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7e278-0265-4251-82cc-5e4ddf4c6276 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:12.651836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:12.658306 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:12.697731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d60e6-216a-4788-9fff-2ecbc2667e31 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:12.700195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7e278-0265-4251-82cc-5e4ddf4c6276 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:12.731008 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb30114-a466-4796-a573-95b47fa37e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:12.731568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb30114-a466-4796-a573-95b47fa37e3f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:12.731848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb30114-a466-4796-a573-95b47fa37e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:12.732123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb30114-a466-4796-a573-95b47fa37e3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:12.733230 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb30114-a466-4796-a573-95b47fa37e3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:12.733667 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e37a8-3cdb-4777-8a42-1521ab15dc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:12.734298 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e37a8-3cdb-4777-8a42-1521ab15dc9a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:12.734624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e37a8-3cdb-4777-8a42-1521ab15dc9a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:12.734708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb30114-a466-4796-a573-95b47fa37e3f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:12.734894 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e37a8-3cdb-4777-8a42-1521ab15dc9a None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:12.735544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e37a8-3cdb-4777-8a42-1521ab15dc9a None admin] RBAC: Adding query filter params (name=member) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:12.736569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e37a8-3cdb-4777-8a42-1521ab15dc9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:12.741399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 42/80] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:12.741399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb30114-a466-4796-a573-95b47fa37e3f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:12.741605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e37a8-3cdb-4777-8a42-1521ab15dc9a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:12.749920 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 39/81] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:12.750790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e37a8-3cdb-4777-8a42-1521ab15dc9a None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:12.834709 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb30114-a466-4796-a573-95b47fa37e3f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:12.850801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e37a8-3cdb-4777-8a42-1521ab15dc9a None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:12.859907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:12.865333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:12.867852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7443fd-31ac-4f54-af58-337e2c325d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:12.868471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7443fd-31ac-4f54-af58-337e2c325d31 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:12.868749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7443fd-31ac-4f54-af58-337e2c325d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:12.869013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7443fd-31ac-4f54-af58-337e2c325d31 None admin] PATH_INFO: `/v3/users/2ebd28e621914177981b750fe6a7566b` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:12.870144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7443fd-31ac-4f54-af58-337e2c325d31 None admin] RBAC: Authorizing `identity:get_user(user_id=2ebd28e621914177981b750fe6a7566b)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:12.871565 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7443fd-31ac-4f54-af58-337e2c325d31 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:12.873121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 43/82] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:12 2022] GET /identity/v3/users/2ebd28e621914177981b750fe6a7566b => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:12.873885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7443fd-31ac-4f54-af58-337e2c325d31 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:12.885400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa31297-2a2b-4fb3-beb9-52c5c08b846a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:12.886076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa31297-2a2b-4fb3-beb9-52c5c08b846a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:12.886337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa31297-2a2b-4fb3-beb9-52c5c08b846a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:12.886607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa31297-2a2b-4fb3-beb9-52c5c08b846a None admin] PATH_INFO: `/v3/users/fd426537f3dc4593941437dc64e7c961` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:12.887742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa31297-2a2b-4fb3-beb9-52c5c08b846a None admin] RBAC: Authorizing `identity:get_user(user_id=fd426537f3dc4593941437dc64e7c961)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:12.889221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa31297-2a2b-4fb3-beb9-52c5c08b846a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:12.890821 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 40/83] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:12 2022] GET /identity/v3/users/fd426537f3dc4593941437dc64e7c961 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:12.895818 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa31297-2a2b-4fb3-beb9-52c5c08b846a None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:12.976470 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7443fd-31ac-4f54-af58-337e2c325d31 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:12.991652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa31297-2a2b-4fb3-beb9-52c5c08b846a None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:13.011420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b9e1b-7b78-41af-9f1e-0610a39a5dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:13.012271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b9e1b-7b78-41af-9f1e-0610a39a5dff None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:13.012614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b9e1b-7b78-41af-9f1e-0610a39a5dff None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:13.012986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b9e1b-7b78-41af-9f1e-0610a39a5dff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:13.026173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4e7a5-c67e-4f4e-ba03-905bd8fc930b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:13.026763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4e7a5-c67e-4f4e-ba03-905bd8fc930b None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:13.027035 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4e7a5-c67e-4f4e-ba03-905bd8fc930b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:13.027297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4e7a5-c67e-4f4e-ba03-905bd8fc930b None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:13.027939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4e7a5-c67e-4f4e-ba03-905bd8fc930b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:13.029353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4e7a5-c67e-4f4e-ba03-905bd8fc930b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:13.073171 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:13.079320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b9e1b-7b78-41af-9f1e-0610a39a5dff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:13.080885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b9e1b-7b78-41af-9f1e-0610a39a5dff None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:13.134474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 41/84] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:23:12 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:13.135292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4e7a5-c67e-4f4e-ba03-905bd8fc930b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:13.183659 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:13.184673 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 44/85] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:13.185700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b9e1b-7b78-41af-9f1e-0610a39a5dff None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:13.253930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4e7a5-c67e-4f4e-ba03-905bd8fc930b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:13.291800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218aef02-576e-4ebd-b100-3e00d0d83a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:13.292446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aef02-576e-4ebd-b100-3e00d0d83a12 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:13.292740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aef02-576e-4ebd-b100-3e00d0d83a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:13.293008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218aef02-576e-4ebd-b100-3e00d0d83a12 None admin] PATH_INFO: `/v3/users/fd426537f3dc4593941437dc64e7c961` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:13.294212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218aef02-576e-4ebd-b100-3e00d0d83a12 None admin] RBAC: Authorizing `identity:get_user(user_id=fd426537f3dc4593941437dc64e7c961)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:13.295671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218aef02-576e-4ebd-b100-3e00d0d83a12 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:13.302810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 42/86] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:13 2022] GET /identity/v3/users/fd426537f3dc4593941437dc64e7c961 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:13.302810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218aef02-576e-4ebd-b100-3e00d0d83a12 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:13.317783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:13.324481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b9e1b-7b78-41af-9f1e-0610a39a5dff None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:13.364344 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03a955-74e1-4ee3-b2ce-bd28231a5a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:13.364966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03a955-74e1-4ee3-b2ce-bd28231a5a87 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:13.365246 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03a955-74e1-4ee3-b2ce-bd28231a5a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:13.365518 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03a955-74e1-4ee3-b2ce-bd28231a5a87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:13.431698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218aef02-576e-4ebd-b100-3e00d0d83a12 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:13.441988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:13.455796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03a955-74e1-4ee3-b2ce-bd28231a5a87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:13.456831 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03a955-74e1-4ee3-b2ce-bd28231a5a87 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:13.474196 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b9b72-56be-4d5c-9446-4424fc0f6f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:13.475189 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b9b72-56be-4d5c-9446-4424fc0f6f50 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:13.475471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b9b72-56be-4d5c-9446-4424fc0f6f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:13.475899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b9b72-56be-4d5c-9446-4424fc0f6f50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:13.558319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b9b72-56be-4d5c-9446-4424fc0f6f50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:13.559997 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b9b72-56be-4d5c-9446-4424fc0f6f50 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:13.582508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:13.593194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 45/87] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:13.594322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03a955-74e1-4ee3-b2ce-bd28231a5a87 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:13.689967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 43/88] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:13.691242 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b9b72-56be-4d5c-9446-4424fc0f6f50 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:13.701196 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:13.713798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03a955-74e1-4ee3-b2ce-bd28231a5a87 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:13.751847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea50bc-e4a4-4595-ba82-a702d3afe47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:13.752727 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea50bc-e4a4-4595-ba82-a702d3afe47b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:13.752902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea50bc-e4a4-4595-ba82-a702d3afe47b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:13.753136 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea50bc-e4a4-4595-ba82-a702d3afe47b None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:13.754723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea50bc-e4a4-4595-ba82-a702d3afe47b None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:13.756183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea50bc-e4a4-4595-ba82-a702d3afe47b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:13.757957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 46/89] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:13 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:13.758725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea50bc-e4a4-4595-ba82-a702d3afe47b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:13.811686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b9b72-56be-4d5c-9446-4424fc0f6f50 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:13.849303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:13.858950 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d3e6d-5c96-4ba8-9669-1ab301848502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:13.859562 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d3e6d-5c96-4ba8-9669-1ab301848502 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:13.859839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d3e6d-5c96-4ba8-9669-1ab301848502 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:13.860561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d3e6d-5c96-4ba8-9669-1ab301848502 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:13.883723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea50bc-e4a4-4595-ba82-a702d3afe47b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:13.934794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e031c2a-17df-47fb-9a6c-6279a45bcd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:13.935664 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e031c2a-17df-47fb-9a6c-6279a45bcd0f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:13.936112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e031c2a-17df-47fb-9a6c-6279a45bcd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:13.936547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e031c2a-17df-47fb-9a6c-6279a45bcd0f None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:13.937975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e031c2a-17df-47fb-9a6c-6279a45bcd0f None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:13.939730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e031c2a-17df-47fb-9a6c-6279a45bcd0f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:13.941072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d3e6d-5c96-4ba8-9669-1ab301848502 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:13.941687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 47/90] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:13 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:13.942626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e031c2a-17df-47fb-9a6c-6279a45bcd0f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:13.943112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d3e6d-5c96-4ba8-9669-1ab301848502 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:13.951532 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:14.062578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 44/91] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:14.068970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d3e6d-5c96-4ba8-9669-1ab301848502 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:14.074315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e031c2a-17df-47fb-9a6c-6279a45bcd0f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:14.117070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:14.129649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b338c2-299c-48b3-a52e-c7f41abe9f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:14.130651 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b338c2-299c-48b3-a52e-c7f41abe9f78 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:14.130939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b338c2-299c-48b3-a52e-c7f41abe9f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:14.131257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b338c2-299c-48b3-a52e-c7f41abe9f78 None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:14.132677 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b338c2-299c-48b3-a52e-c7f41abe9f78 None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:14.134889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b338c2-299c-48b3-a52e-c7f41abe9f78 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:14.137497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 48/92] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:13 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:14.142465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b338c2-299c-48b3-a52e-c7f41abe9f78 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:14.179297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d3e6d-5c96-4ba8-9669-1ab301848502 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:14.187534 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:14.217287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a374360-8f4c-4b7a-9075-caccd9b8fb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:14.218214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a374360-8f4c-4b7a-9075-caccd9b8fb20 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:14.218659 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a374360-8f4c-4b7a-9075-caccd9b8fb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:14.219121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a374360-8f4c-4b7a-9075-caccd9b8fb20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:14.220542 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a374360-8f4c-4b7a-9075-caccd9b8fb20 None admin] RBAC: Adding query filter params (role.id=413cc5f8baab440bab827a3547b65f33, scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e, user.id=fd426537f3dc4593941437dc64e7c961) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:14.221163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a374360-8f4c-4b7a-9075-caccd9b8fb20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:14.222955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a374360-8f4c-4b7a-9075-caccd9b8fb20 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:14.229166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 45/93] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:14 2022] GET /identity/v3/role_assignments?user.id=fd426537f3dc4593941437dc64e7c961&scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e&role.id=413cc5f8baab440bab827a3547b65f33 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:14.230203 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a374360-8f4c-4b7a-9075-caccd9b8fb20 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:14.295425 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b338c2-299c-48b3-a52e-c7f41abe9f78 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:14.387501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a374360-8f4c-4b7a-9075-caccd9b8fb20 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:14.399562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eef71b-c2b6-4dbc-adee-29d8c49fa14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:14.400233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eef71b-c2b6-4dbc-adee-29d8c49fa14f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:14.400463 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eef71b-c2b6-4dbc-adee-29d8c49fa14f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:14.400738 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eef71b-c2b6-4dbc-adee-29d8c49fa14f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:14.401472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eef71b-c2b6-4dbc-adee-29d8c49fa14f None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e, user.id=2ebd28e621914177981b750fe6a7566b) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:14.401776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eef71b-c2b6-4dbc-adee-29d8c49fa14f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:14.403269 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eef71b-c2b6-4dbc-adee-29d8c49fa14f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:14.417613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 49/94] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:14 2022] GET /identity/v3/role_assignments?user.id=2ebd28e621914177981b750fe6a7566b&scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:14.418979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eef71b-c2b6-4dbc-adee-29d8c49fa14f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:14.452911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a40123-ae85-4285-9f9d-329642be63db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:14.454262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a40123-ae85-4285-9f9d-329642be63db None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:14.455189 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a40123-ae85-4285-9f9d-329642be63db None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:14.457112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a40123-ae85-4285-9f9d-329642be63db None admin] PATH_INFO: `/v3/projects/01f11c3472b641ea82fb9a5f60117973` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:14.458755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a40123-ae85-4285-9f9d-329642be63db None admin] RBAC: Authorizing `identity:get_project(project_id=01f11c3472b641ea82fb9a5f60117973)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:14.460425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a40123-ae85-4285-9f9d-329642be63db None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:14.462725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 46/95] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:14 2022] GET /identity/v3/projects/01f11c3472b641ea82fb9a5f60117973 => generated 312 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:14.468972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a40123-ae85-4285-9f9d-329642be63db None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:14.518467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:14.584110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a40123-ae85-4285-9f9d-329642be63db None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:14.602208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:14.658992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f519a19-b8a6-42da-a215-ce4e1f23dfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:14.659540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519a19-b8a6-42da-a215-ce4e1f23dfc4 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:14.659819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519a19-b8a6-42da-a215-ce4e1f23dfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:14.660098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f519a19-b8a6-42da-a215-ce4e1f23dfc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:14.660891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f519a19-b8a6-42da-a215-ce4e1f23dfc4 None admin] RBAC: Adding query filter params (role.id=413cc5f8baab440bab827a3547b65f33, scope.project.id=01f11c3472b641ea82fb9a5f60117973, user.id=fd426537f3dc4593941437dc64e7c961) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:14.661199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f519a19-b8a6-42da-a215-ce4e1f23dfc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:14.662694 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f519a19-b8a6-42da-a215-ce4e1f23dfc4 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:14.666184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eef71b-c2b6-4dbc-adee-29d8c49fa14f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:14.668940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 47/96] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:14 2022] GET /identity/v3/role_assignments?user.id=fd426537f3dc4593941437dc64e7c961&scope.project.id=01f11c3472b641ea82fb9a5f60117973&role.id=413cc5f8baab440bab827a3547b65f33 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:14.725686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b8456-3d3e-4c7f-8b23-c23d2e4f049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:14.726329 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b8456-3d3e-4c7f-8b23-c23d2e4f049b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:14.726607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b8456-3d3e-4c7f-8b23-c23d2e4f049b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:14.726875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b8456-3d3e-4c7f-8b23-c23d2e4f049b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:14.727610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b8456-3d3e-4c7f-8b23-c23d2e4f049b None admin] RBAC: Adding query filter params (role.id=93f699f03f0f41498415ab9533bdfcb4, scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e, user.id=fd426537f3dc4593941437dc64e7c961) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:14.727916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b8456-3d3e-4c7f-8b23-c23d2e4f049b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:14.732152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b8456-3d3e-4c7f-8b23-c23d2e4f049b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:14.738736 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 50/97] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:14 2022] GET /identity/v3/role_assignments?user.id=fd426537f3dc4593941437dc64e7c961&scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e&role.id=93f699f03f0f41498415ab9533bdfcb4 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:15.848642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77030b-7ca9-4e42-9e93-a346e88f93f9 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:15.849752 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77030b-7ca9-4e42-9e93-a346e88f93f9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:15.850286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77030b-7ca9-4e42-9e93-a346e88f93f9 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:15.851807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 48/98] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:15.857186 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd2f6f-bd15-4570-b06d-a2193de4de46 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:15.857986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd2f6f-bd15-4570-b06d-a2193de4de46 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:15.858455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd2f6f-bd15-4570-b06d-a2193de4de46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:15.884181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-55dd2f6f-bd15-4570-b06d-a2193de4de46 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:15.885729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-55dd2f6f-bd15-4570-b06d-a2193de4de46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:15.972366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 51/99] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:16.235151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbb2a7-f653-41be-bdc7-dbc9192aef7c None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:16.235843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbb2a7-f653-41be-bdc7-dbc9192aef7c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:16.236343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdbb2a7-f653-41be-bdc7-dbc9192aef7c None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:16.238207 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 49/100] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:16.243166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00270ecd-99c5-4335-a645-2f03a05e7960 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:16.243652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00270ecd-99c5-4335-a645-2f03a05e7960 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:16.244110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00270ecd-99c5-4335-a645-2f03a05e7960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:16.268669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-00270ecd-99c5-4335-a645-2f03a05e7960 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:16.270186 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-00270ecd-99c5-4335-a645-2f03a05e7960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:16.332555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:16.351181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 52/101] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:16.402485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbb2a7-f653-41be-bdc7-dbc9192aef7c None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:16.522703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdbb2a7-f653-41be-bdc7-dbc9192aef7c None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:16.525576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:16.566008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6fd24-4fb1-4443-b7e0-2105f5afcbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:16.566647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6fd24-4fb1-4443-b7e0-2105f5afcbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:16.566917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6fd24-4fb1-4443-b7e0-2105f5afcbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:16.567182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6fd24-4fb1-4443-b7e0-2105f5afcbf6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:16.568291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6fd24-4fb1-4443-b7e0-2105f5afcbf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:16.569797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6fd24-4fb1-4443-b7e0-2105f5afcbf6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:16.571599 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 50/102] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:16.576903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00270ecd-99c5-4335-a645-2f03a05e7960 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:16.661240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6fd24-4fb1-4443-b7e0-2105f5afcbf6 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:16.716906 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00270ecd-99c5-4335-a645-2f03a05e7960 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:16.779253 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77898764-eab1-4976-9d40-2fbdb4a4c84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:16.780233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77898764-eab1-4976-9d40-2fbdb4a4c84e None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:16.780690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77898764-eab1-4976-9d40-2fbdb4a4c84e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:16.781139 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77898764-eab1-4976-9d40-2fbdb4a4c84e None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:16.782118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77898764-eab1-4976-9d40-2fbdb4a4c84e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:16.789853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77898764-eab1-4976-9d40-2fbdb4a4c84e None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:16.797855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6fd24-4fb1-4443-b7e0-2105f5afcbf6 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:16.834738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf8919-1bc7-4846-8ca1-e45aa7e29eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:16.835482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf8919-1bc7-4846-8ca1-e45aa7e29eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:16.835762 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf8919-1bc7-4846-8ca1-e45aa7e29eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:16.837493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf8919-1bc7-4846-8ca1-e45aa7e29eb9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:16.846790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf8919-1bc7-4846-8ca1-e45aa7e29eb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-ebaf8919-1bc7-4846-8ca1-e45aa7e29eb9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:16.852313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:16.853597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:16.854888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:16.854888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:16.854888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:16.854888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:16.854888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:16.854888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:16.854888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:16.854888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:16.854888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:16.854888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:16.854888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:16.854888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:16.854888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 51/103] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:23:16.859820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf8919-1bc7-4846-8ca1-e45aa7e29eb9 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:16.899992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:16.910517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 53/104] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:23:16 2022] POST /identity/v3/projects => generated 302 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:17.013510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf8919-1bc7-4846-8ca1-e45aa7e29eb9 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:17.065085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d50898-a459-4e56-9582-74d10db64a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:17.065539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50898-a459-4e56-9582-74d10db64a0d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:17.065852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50898-a459-4e56-9582-74d10db64a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:17.066108 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50898-a459-4e56-9582-74d10db64a0d None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:17.066780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d50898-a459-4e56-9582-74d10db64a0d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:17.067082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d50898-a459-4e56-9582-74d10db64a0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:17.068459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d50898-a459-4e56-9582-74d10db64a0d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:17.082569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 52/105] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:17.088928 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77898764-eab1-4976-9d40-2fbdb4a4c84e None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:17.172408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:17.245041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77898764-eab1-4976-9d40-2fbdb4a4c84e None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:17.286365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678260fb-32d2-4d25-9ff5-42aa48fe927f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:17.287005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678260fb-32d2-4d25-9ff5-42aa48fe927f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:17.287283 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678260fb-32d2-4d25-9ff5-42aa48fe927f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:17.287555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678260fb-32d2-4d25-9ff5-42aa48fe927f None admin] PATH_INFO: `/v3/users/fc2bbe3ff1b049a7af86aaa72244780c` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:17.311744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678260fb-32d2-4d25-9ff5-42aa48fe927f None admin] RBAC: Authorizing `identity:get_user(user_id=fc2bbe3ff1b049a7af86aaa72244780c)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:17.313198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678260fb-32d2-4d25-9ff5-42aa48fe927f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:17.315719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 54/106] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:17 2022] GET /identity/v3/users/fc2bbe3ff1b049a7af86aaa72244780c => generated 289 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:17.322180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d50898-a459-4e56-9582-74d10db64a0d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:17.465326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:17.474561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d50898-a459-4e56-9582-74d10db64a0d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:17.533453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b733328-01c7-4c06-8c08-edb065481e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:17.534110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b733328-01c7-4c06-8c08-edb065481e11 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:17.534388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b733328-01c7-4c06-8c08-edb065481e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:17.534658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b733328-01c7-4c06-8c08-edb065481e11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:17.620535 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b733328-01c7-4c06-8c08-edb065481e11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:17.622031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b733328-01c7-4c06-8c08-edb065481e11 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:17.684966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d0e05-f7bf-42ed-8eac-5f5b68f1f2a6 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:17.685320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d0e05-f7bf-42ed-8eac-5f5b68f1f2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:17.685507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d0e05-f7bf-42ed-8eac-5f5b68f1f2a6 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:17.686879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 55/107] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:23:17.706677 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d9e8e-4d1c-4a9d-98ae-116bdecadfa9 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:17.706998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d9e8e-4d1c-4a9d-98ae-116bdecadfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:17.707227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d9e8e-4d1c-4a9d-98ae-116bdecadfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:17.742920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-849d9e8e-4d1c-4a9d-98ae-116bdecadfa9 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:17.745441 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-849d9e8e-4d1c-4a9d-98ae-116bdecadfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:17.763544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:17.783974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 53/108] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:17.789544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46302741-2d4a-4b7c-bd92-df12b73d306c None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:17.794904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46302741-2d4a-4b7c-bd92-df12b73d306c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:17.795539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46302741-2d4a-4b7c-bd92-df12b73d306c None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:17.797350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 54/109] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:17.798562 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46302741-2d4a-4b7c-bd92-df12b73d306c None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:17.888387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 56/110] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:17.890017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef2287-7037-4986-a98e-f62e2cc479a7 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:17.890473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef2287-7037-4986-a98e-f62e2cc479a7 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:17.890912 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef2287-7037-4986-a98e-f62e2cc479a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:17.942854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-26ef2287-7037-4986-a98e-f62e2cc479a7 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:17.944700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-26ef2287-7037-4986-a98e-f62e2cc479a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:17.959048 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46302741-2d4a-4b7c-bd92-df12b73d306c None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:17.992158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:18.021264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af1a32-afce-4c22-ad47-56b6427b579c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:18.021952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af1a32-afce-4c22-ad47-56b6427b579c None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:18.022209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af1a32-afce-4c22-ad47-56b6427b579c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:18.023569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af1a32-afce-4c22-ad47-56b6427b579c None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:18.025497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af1a32-afce-4c22-ad47-56b6427b579c None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:18.027081 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af1a32-afce-4c22-ad47-56b6427b579c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:18.029056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 55/111] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:17 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:18.035420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be7049-34e7-4087-a891-05cc10241415 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:18.035704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be7049-34e7-4087-a891-05cc10241415 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:18.035979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be7049-34e7-4087-a891-05cc10241415 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:18.037656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 56/112] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:18.040095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be7049-34e7-4087-a891-05cc10241415 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:18.086207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 57/113] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:18.089044 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062722e8-a603-44e0-b63c-0da53e1696b4 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:18.089329 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062722e8-a603-44e0-b63c-0da53e1696b4 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:18.089608 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062722e8-a603-44e0-b63c-0da53e1696b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:18.124813 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-062722e8-a603-44e0-b63c-0da53e1696b4 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:18.127110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-062722e8-a603-44e0-b63c-0da53e1696b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:18.166287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:18.250098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be7049-34e7-4087-a891-05cc10241415 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:18.253018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 58/114] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:18.253720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062722e8-a603-44e0-b63c-0da53e1696b4 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:18.335955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2092137-cc2d-47e0-8e50-a7a62be816a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:18.336584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2092137-cc2d-47e0-8e50-a7a62be816a4 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:18.336864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2092137-cc2d-47e0-8e50-a7a62be816a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:18.337135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2092137-cc2d-47e0-8e50-a7a62be816a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:18.342679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2092137-cc2d-47e0-8e50-a7a62be816a4 None admin] RBAC: Adding query filter params (role.id=c8fdda710e4f416380f2e8d61fa5b71f, scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e, user.id=fc2bbe3ff1b049a7af86aaa72244780c) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:18.343128 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2092137-cc2d-47e0-8e50-a7a62be816a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:18.344638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2092137-cc2d-47e0-8e50-a7a62be816a4 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:18.367942 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 57/115] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:18 2022] GET /identity/v3/role_assignments?user.id=fc2bbe3ff1b049a7af86aaa72244780c&scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e&role.id=c8fdda710e4f416380f2e8d61fa5b71f => generated 259 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:18.429648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2092137-cc2d-47e0-8e50-a7a62be816a4 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:18.441385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:18.464936 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062722e8-a603-44e0-b63c-0da53e1696b4 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:18.553918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a93ada-8184-4d9d-9686-cfc9ddf3e0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:18.554577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a93ada-8184-4d9d-9686-cfc9ddf3e0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:18.554851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a93ada-8184-4d9d-9686-cfc9ddf3e0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:18.555118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a93ada-8184-4d9d-9686-cfc9ddf3e0c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:18.578989 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a93ada-8184-4d9d-9686-cfc9ddf3e0c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-e2a93ada-8184-4d9d-9686-cfc9ddf3e0c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:18.586920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:18.587860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:18.588644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:18.588644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:18.588644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:18.588644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:18.588644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:18.588644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:18.588644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:18.588644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:18.588644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:18.588644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:18.588644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:18.588644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:18.588644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 59/116] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:18.590279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3cde8-28de-43a0-8b67-c4812202b2c1 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:18.592400 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3cde8-28de-43a0-8b67-c4812202b2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:18.596464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3cde8-28de-43a0-8b67-c4812202b2c1 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:18.597564 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 60/117] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:18.605610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3cde8-28de-43a0-8b67-c4812202b2c1 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:18.627431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2092137-cc2d-47e0-8e50-a7a62be816a4 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:18.689772 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:18.700832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb55bf4-e5e5-44de-bcfd-37bacb5e3c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:18.701465 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb55bf4-e5e5-44de-bcfd-37bacb5e3c33 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:18.705462 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb55bf4-e5e5-44de-bcfd-37bacb5e3c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:18.705462 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb55bf4-e5e5-44de-bcfd-37bacb5e3c33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:18.713338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb55bf4-e5e5-44de-bcfd-37bacb5e3c33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-feb55bf4-e5e5-44de-bcfd-37bacb5e3c33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:18.718565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:18.719400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:18.721272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:18.721272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:18.721272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:18.721272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:18.721272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:18.721272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:18.721272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:18.721272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:18.721272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:18.721272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:18.721272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:18.721272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:18.721272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 58/118] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:23:18.721937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedd7dc-8699-4da2-bf7b-c9d175e137d3 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:18.721937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedd7dc-8699-4da2-bf7b-c9d175e137d3 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:18.726616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedd7dc-8699-4da2-bf7b-c9d175e137d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:18.747696 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-bbedd7dc-8699-4da2-bf7b-c9d175e137d3 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:18.748874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-bbedd7dc-8699-4da2-bf7b-c9d175e137d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:18.785232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3cde8-28de-43a0-8b67-c4812202b2c1 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:18.836199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b3ffa-f43e-4448-a9b5-2850a436e7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:18.836860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b3ffa-f43e-4448-a9b5-2850a436e7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:18.837141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b3ffa-f43e-4448-a9b5-2850a436e7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:18.837412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b3ffa-f43e-4448-a9b5-2850a436e7d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:18.838114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b3ffa-f43e-4448-a9b5-2850a436e7d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:18.838398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b3ffa-f43e-4448-a9b5-2850a436e7d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:18.839761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b3ffa-f43e-4448-a9b5-2850a436e7d7 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:18.840830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 59/119] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:18.842214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedd7dc-8699-4da2-bf7b-c9d175e137d3 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:18.850803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 61/120] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:18.851655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b3ffa-f43e-4448-a9b5-2850a436e7d7 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:18.867050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:18.976021 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b3ffa-f43e-4448-a9b5-2850a436e7d7 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:19.000098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedd7dc-8699-4da2-bf7b-c9d175e137d3 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:19.010166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0710bad-f6ee-405c-bc41-680a852501fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:19.010862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0710bad-f6ee-405c-bc41-680a852501fa None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:19.011224 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0710bad-f6ee-405c-bc41-680a852501fa None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:19.011561 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0710bad-f6ee-405c-bc41-680a852501fa None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:19.012285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0710bad-f6ee-405c-bc41-680a852501fa None admin] RBAC: Adding query filter params (name=member) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:19.012650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0710bad-f6ee-405c-bc41-680a852501fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:19.014104 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0710bad-f6ee-405c-bc41-680a852501fa None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:19.023398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 62/121] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:19.024277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0710bad-f6ee-405c-bc41-680a852501fa None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:19.029220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:19.063983 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf502dc-6d91-4591-ab0b-1fba9921ddfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:19.064627 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf502dc-6d91-4591-ab0b-1fba9921ddfd None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:19.070012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf502dc-6d91-4591-ab0b-1fba9921ddfd None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:19.070012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf502dc-6d91-4591-ab0b-1fba9921ddfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:19.086633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf502dc-6d91-4591-ab0b-1fba9921ddfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-0cf502dc-6d91-4591-ab0b-1fba9921ddfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:19.091590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:19.092451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:19.093626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:19.093626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:19.093626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:19.093626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:19.093626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:19.093626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:19.093626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:19.093626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:19.093626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:19.093626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:19.093626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:19.093626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:19.093626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 60/122] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:23:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:19.105376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:19.111012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf502dc-6d91-4591-ab0b-1fba9921ddfd None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:19.138916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0710bad-f6ee-405c-bc41-680a852501fa None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:19.173856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e91ac2-6d87-41ed-9b42-33807757adf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:19.174548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e91ac2-6d87-41ed-9b42-33807757adf0 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:19.174824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e91ac2-6d87-41ed-9b42-33807757adf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:19.175094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e91ac2-6d87-41ed-9b42-33807757adf0 None admin] PATH_INFO: `/v3/users/fd426537f3dc4593941437dc64e7c961` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:19.176286 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e91ac2-6d87-41ed-9b42-33807757adf0 None admin] RBAC: Authorizing `identity:get_user(user_id=fd426537f3dc4593941437dc64e7c961)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:19.177731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e91ac2-6d87-41ed-9b42-33807757adf0 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:19.179532 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 63/123] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:19 2022] GET /identity/v3/users/fd426537f3dc4593941437dc64e7c961 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:19.180078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e91ac2-6d87-41ed-9b42-33807757adf0 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:19.238616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf502dc-6d91-4591-ab0b-1fba9921ddfd None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:19.275613 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814de900-0d80-44e9-8370-85ee6478fda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:19.276352 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814de900-0d80-44e9-8370-85ee6478fda3 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:19.276638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814de900-0d80-44e9-8370-85ee6478fda3 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:19.276913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814de900-0d80-44e9-8370-85ee6478fda3 None admin] PATH_INFO: `/v3/users/fd426537f3dc4593941437dc64e7c961` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:19.278225 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814de900-0d80-44e9-8370-85ee6478fda3 None admin] RBAC: Authorizing `identity:get_user(user_id=fd426537f3dc4593941437dc64e7c961)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:19.279792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814de900-0d80-44e9-8370-85ee6478fda3 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:19.281703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 61/124] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:19 2022] GET /identity/v3/users/fd426537f3dc4593941437dc64e7c961 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:19.282456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814de900-0d80-44e9-8370-85ee6478fda3 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:19.289474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e91ac2-6d87-41ed-9b42-33807757adf0 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:19.324871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52a740-3625-4651-b589-5687b39c5583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:19.325487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52a740-3625-4651-b589-5687b39c5583 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:19.325764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52a740-3625-4651-b589-5687b39c5583 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:19.326060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52a740-3625-4651-b589-5687b39c5583 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:19.326736 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52a740-3625-4651-b589-5687b39c5583 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:19.327034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52a740-3625-4651-b589-5687b39c5583 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:19.328467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52a740-3625-4651-b589-5687b39c5583 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:19.331557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:19.337174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 64/125] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:19.342668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52a740-3625-4651-b589-5687b39c5583 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:19.346382 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:19.388970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814de900-0d80-44e9-8370-85ee6478fda3 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:19.427618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c4286-00f3-41de-8a79-e72a4eb4278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:19.428586 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c4286-00f3-41de-8a79-e72a4eb4278c None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:19.429041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c4286-00f3-41de-8a79-e72a4eb4278c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:19.429457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c4286-00f3-41de-8a79-e72a4eb4278c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:19.441218 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c4286-00f3-41de-8a79-e72a4eb4278c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-a98c4286-00f3-41de-8a79-e72a4eb4278c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:19.446577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:19.447632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:19.448609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:19.448609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:19.448609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:19.448609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:19.448609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:19.448609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:19.448609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:19.448609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:19.448609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:19.448609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:19.448609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:19.448609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:19.449238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52a740-3625-4651-b589-5687b39c5583 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:19.450638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 62/126] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:23:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:19.451553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c4286-00f3-41de-8a79-e72a4eb4278c None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:19.484187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3fa62-503b-4e2e-b123-d25dece00575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:19.484833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3fa62-503b-4e2e-b123-d25dece00575 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:19.485159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3fa62-503b-4e2e-b123-d25dece00575 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:19.485434 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3fa62-503b-4e2e-b123-d25dece00575 None admin] PATH_INFO: `/v3/projects/01f11c3472b641ea82fb9a5f60117973` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:19.487235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3fa62-503b-4e2e-b123-d25dece00575 None admin] RBAC: Authorizing `identity:get_project(project_id=01f11c3472b641ea82fb9a5f60117973)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:19.488874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a3fa62-503b-4e2e-b123-d25dece00575 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:19.490662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 65/127] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:19 2022] GET /identity/v3/projects/01f11c3472b641ea82fb9a5f60117973 => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:19.491493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3fa62-503b-4e2e-b123-d25dece00575 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:19.566627 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:19.570163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c4286-00f3-41de-8a79-e72a4eb4278c None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:19.571805 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:19.598556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3fa62-503b-4e2e-b123-d25dece00575 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:19.606731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b578750-de8b-4864-9950-bb12d052b938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:19.607546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b578750-de8b-4864-9950-bb12d052b938 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:19.607980 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b578750-de8b-4864-9950-bb12d052b938 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:19.608468 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b578750-de8b-4864-9950-bb12d052b938 None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:19.609854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b578750-de8b-4864-9950-bb12d052b938 None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:19.612043 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b578750-de8b-4864-9950-bb12d052b938 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:19.614429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 63/128] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:19 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:19.615384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b578750-de8b-4864-9950-bb12d052b938 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:19.640336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75078c99-94ea-4101-b106-291192b19348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:19.640965 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75078c99-94ea-4101-b106-291192b19348 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:19.641664 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75078c99-94ea-4101-b106-291192b19348 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:19.641911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75078c99-94ea-4101-b106-291192b19348 None admin] PATH_INFO: `/v3/users/2ebd28e621914177981b750fe6a7566b` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:19.643568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75078c99-94ea-4101-b106-291192b19348 None admin] RBAC: Authorizing `identity:get_user(user_id=2ebd28e621914177981b750fe6a7566b)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:19.645570 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75078c99-94ea-4101-b106-291192b19348 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:19.648111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 66/129] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:19 2022] GET /identity/v3/users/2ebd28e621914177981b750fe6a7566b => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:19.648870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75078c99-94ea-4101-b106-291192b19348 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:19.739409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b578750-de8b-4864-9950-bb12d052b938 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:19.783688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43778417-ed84-435c-8294-f68e5332559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:19.784270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43778417-ed84-435c-8294-f68e5332559b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:19.784556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43778417-ed84-435c-8294-f68e5332559b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:19.784830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43778417-ed84-435c-8294-f68e5332559b None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:19.785510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43778417-ed84-435c-8294-f68e5332559b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:19.785803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43778417-ed84-435c-8294-f68e5332559b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:19.787352 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43778417-ed84-435c-8294-f68e5332559b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:19.787937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75078c99-94ea-4101-b106-291192b19348 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:19.803986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 64/130] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:23:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:19.803986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43778417-ed84-435c-8294-f68e5332559b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:19.835519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:19.838940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8e217-0bb0-4aa3-9ed3-c1f1857f0585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:19.840403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8e217-0bb0-4aa3-9ed3-c1f1857f0585 None admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:19.840691 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8e217-0bb0-4aa3-9ed3-c1f1857f0585 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:19.840972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8e217-0bb0-4aa3-9ed3-c1f1857f0585 None admin] PATH_INFO: `/v3/projects/01f11c3472b641ea82fb9a5f60117973/users/fd426537f3dc4593941437dc64e7c961/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:19.842954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8e217-0bb0-4aa3-9ed3-c1f1857f0585 None admin] RBAC: Authorizing `identity:create_grant(project_id=01f11c3472b641ea82fb9a5f60117973, user_id=fd426537f3dc4593941437dc64e7c961, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:19.844811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a8e217-0bb0-4aa3-9ed3-c1f1857f0585 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:19.916218 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43778417-ed84-435c-8294-f68e5332559b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:19.951477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa188461-2e41-48a5-b584-c08c2f22bee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:19.952109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa188461-2e41-48a5-b584-c08c2f22bee3 None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:19.952440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa188461-2e41-48a5-b584-c08c2f22bee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:19.952709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa188461-2e41-48a5-b584-c08c2f22bee3 None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e/users/fd426537f3dc4593941437dc64e7c961/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:19.955152 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa188461-2e41-48a5-b584-c08c2f22bee3 None admin] RBAC: Authorizing `identity:create_grant(project_id=703ed0f0d1ff4514b6dc0e857cbf989e, user_id=fd426537f3dc4593941437dc64e7c961, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:19.955795 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:19.957033 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa188461-2e41-48a5-b584-c08c2f22bee3 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:19.960626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 67/131] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:23:19 2022] PUT /identity/v3/projects/01f11c3472b641ea82fb9a5f60117973/users/fd426537f3dc4593941437dc64e7c961/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:23:19.961813 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8e217-0bb0-4aa3-9ed3-c1f1857f0585 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:20.053045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 65/132] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:23:19 2022] PUT /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e/users/fd426537f3dc4593941437dc64e7c961/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:23:20.054663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa188461-2e41-48a5-b584-c08c2f22bee3 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:20.118331 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a8e217-0bb0-4aa3-9ed3-c1f1857f0585 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:20.179257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a2c84-fb0f-4ad0-8b63-1bce7a96301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:20.179905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a2c84-fb0f-4ad0-8b63-1bce7a96301b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:20.180180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a2c84-fb0f-4ad0-8b63-1bce7a96301b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:20.180450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a2c84-fb0f-4ad0-8b63-1bce7a96301b None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:20.181730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a2c84-fb0f-4ad0-8b63-1bce7a96301b None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:20.184151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a2c84-fb0f-4ad0-8b63-1bce7a96301b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:20.186374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:20.186847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 68/133] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:19 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:20.189320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6987764-45f4-4bd6-8726-c1c29288de22 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:20.189864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6987764-45f4-4bd6-8726-c1c29288de22 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:20.190351 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6987764-45f4-4bd6-8726-c1c29288de22 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:20.192158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 69/134] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:20.193072 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6987764-45f4-4bd6-8726-c1c29288de22 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:20.196678 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa188461-2e41-48a5-b584-c08c2f22bee3 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:20.251878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff8e1de-e5b8-4e79-ac48-aaf7791e6da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:20.252570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8e1de-e5b8-4e79-ac48-aaf7791e6da6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:20.252858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8e1de-e5b8-4e79-ac48-aaf7791e6da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:20.253135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff8e1de-e5b8-4e79-ac48-aaf7791e6da6 None admin] PATH_INFO: `/v3/users/fd426537f3dc4593941437dc64e7c961` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:20.254582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8e1de-e5b8-4e79-ac48-aaf7791e6da6 None admin] RBAC: Authorizing `identity:get_user(user_id=fd426537f3dc4593941437dc64e7c961)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:20.256634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff8e1de-e5b8-4e79-ac48-aaf7791e6da6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:20.261858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 66/135] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:20 2022] GET /identity/v3/users/fd426537f3dc4593941437dc64e7c961 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:20.261858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e6166-75a8-4691-84d4-e91e84f48417 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:20.261858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e6166-75a8-4691-84d4-e91e84f48417 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:20.262321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e6166-75a8-4691-84d4-e91e84f48417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:20.281790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:20.286643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-480e6166-75a8-4691-84d4-e91e84f48417 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:20.288051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-480e6166-75a8-4691-84d4-e91e84f48417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:20.323986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6987764-45f4-4bd6-8726-c1c29288de22 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:20.362547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfcf34e-46f3-4712-bdc9-ca71fc11cf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:20.363555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcf34e-46f3-4712-bdc9-ca71fc11cf59 None admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:20.364063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcf34e-46f3-4712-bdc9-ca71fc11cf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:20.364551 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcf34e-46f3-4712-bdc9-ca71fc11cf59 None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e/users/2ebd28e621914177981b750fe6a7566b/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:20.366679 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfcf34e-46f3-4712-bdc9-ca71fc11cf59 None admin] RBAC: Authorizing `identity:create_grant(project_id=703ed0f0d1ff4514b6dc0e857cbf989e, user_id=2ebd28e621914177981b750fe6a7566b, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:20.369669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfcf34e-46f3-4712-bdc9-ca71fc11cf59 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:20.380406 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 67/136] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:20.380900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e6166-75a8-4691-84d4-e91e84f48417 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:20.447521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 70/137] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:23:20 2022] PUT /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e/users/2ebd28e621914177981b750fe6a7566b/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:23:20.479030 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:20.490227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e6166-75a8-4691-84d4-e91e84f48417 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:20.529271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379719d6-c123-4276-9e94-8f635e2262ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:20.529942 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379719d6-c123-4276-9e94-8f635e2262ca None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:20.530222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379719d6-c123-4276-9e94-8f635e2262ca None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:20.530545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379719d6-c123-4276-9e94-8f635e2262ca None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:20.531683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379719d6-c123-4276-9e94-8f635e2262ca None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:20.533491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379719d6-c123-4276-9e94-8f635e2262ca None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:20.535301 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 68/138] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:20 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:20.541139 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfcf34e-46f3-4712-bdc9-ca71fc11cf59 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:20.670354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfcf34e-46f3-4712-bdc9-ca71fc11cf59 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:20.699325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:20.709766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67e426-d062-4c53-aceb-f60b801c9762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:20.710458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67e426-d062-4c53-aceb-f60b801c9762 None admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:20.710734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67e426-d062-4c53-aceb-f60b801c9762 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:20.711005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67e426-d062-4c53-aceb-f60b801c9762 None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e/users/fd426537f3dc4593941437dc64e7c961/roles/93f699f03f0f41498415ab9533bdfcb4` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:20.713207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67e426-d062-4c53-aceb-f60b801c9762 None admin] RBAC: Authorizing `identity:create_grant(project_id=703ed0f0d1ff4514b6dc0e857cbf989e, user_id=fd426537f3dc4593941437dc64e7c961, role_id=93f699f03f0f41498415ab9533bdfcb4)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:20.714706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67e426-d062-4c53-aceb-f60b801c9762 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:20.805880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 71/139] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:23:20 2022] PUT /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e/users/fd426537f3dc4593941437dc64e7c961/roles/93f699f03f0f41498415ab9533bdfcb4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:23:20.827847 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379719d6-c123-4276-9e94-8f635e2262ca None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:20.961762 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379719d6-c123-4276-9e94-8f635e2262ca None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:21.002570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bac1a-bf18-42fd-be21-7469e9e1b1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:21.002910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bac1a-bf18-42fd-be21-7469e9e1b1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:21.003150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bac1a-bf18-42fd-be21-7469e9e1b1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:21.003425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bac1a-bf18-42fd-be21-7469e9e1b1a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:21.004605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bac1a-bf18-42fd-be21-7469e9e1b1a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:21.006106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bac1a-bf18-42fd-be21-7469e9e1b1a9 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:21.008036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 69/140] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:21.013318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67e426-d062-4c53-aceb-f60b801c9762 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:21.106807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283543c6-a72a-42d2-951a-e2898597f5e0 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:21.107308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283543c6-a72a-42d2-951a-e2898597f5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:21.107380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283543c6-a72a-42d2-951a-e2898597f5e0 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:21.108820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 70/141] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:21.132420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dea7e-a5e4-4c5f-9030-33df73fbf6ad None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:21.132708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dea7e-a5e4-4c5f-9030-33df73fbf6ad None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:21.132984 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05dea7e-a5e4-4c5f-9030-33df73fbf6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:21.164786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-a05dea7e-a5e4-4c5f-9030-33df73fbf6ad None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:21.165878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67e426-d062-4c53-aceb-f60b801c9762 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:21.166347 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a05dea7e-a5e4-4c5f-9030-33df73fbf6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:21.201247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:21.218516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c7d26-c0a3-4bc0-9e9e-8c45fa57e8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:21.219420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c7d26-c0a3-4bc0-9e9e-8c45fa57e8b9 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:21.219901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c7d26-c0a3-4bc0-9e9e-8c45fa57e8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:21.220367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c7d26-c0a3-4bc0-9e9e-8c45fa57e8b9 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:21.221221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c7d26-c0a3-4bc0-9e9e-8c45fa57e8b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:21.222944 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c7d26-c0a3-4bc0-9e9e-8c45fa57e8b9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:21.284742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 71/142] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:21.356305 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 72/143] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:23:21 2022] POST /identity/v3/users => generated 283 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:21.672900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dea7e-a5e4-4c5f-9030-33df73fbf6ad None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:21.784238 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05dea7e-a5e4-4c5f-9030-33df73fbf6ad None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:21.821013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81571996-5f88-4f7d-9525-98a0caeb8fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:21.821649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81571996-5f88-4f7d-9525-98a0caeb8fcd None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:21.821959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81571996-5f88-4f7d-9525-98a0caeb8fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:21.822232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81571996-5f88-4f7d-9525-98a0caeb8fcd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:21.834053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81571996-5f88-4f7d-9525-98a0caeb8fcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-81571996-5f88-4f7d-9525-98a0caeb8fcd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:21.838890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:21.839716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:21.840467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:21.840467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:21.840467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:21.840467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:21.840467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:21.840467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:21.840467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:21.840467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:21.840467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:21.840467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:21.840467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:21.840467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:21.840467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 72/144] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:21.847029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c7d26-c0a3-4bc0-9e9e-8c45fa57e8b9 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:21.905355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:21.983081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c7d26-c0a3-4bc0-9e9e-8c45fa57e8b9 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:22.033703 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c67869-3d64-4f79-95ab-15e9ceb5fb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:22.034357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c67869-3d64-4f79-95ab-15e9ceb5fb59 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:22.034635 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c67869-3d64-4f79-95ab-15e9ceb5fb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:22.034906 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c67869-3d64-4f79-95ab-15e9ceb5fb59 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:22.035582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c67869-3d64-4f79-95ab-15e9ceb5fb59 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:22.035880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c67869-3d64-4f79-95ab-15e9ceb5fb59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:22.037273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c67869-3d64-4f79-95ab-15e9ceb5fb59 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:22.050489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 73/145] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:22.055641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81571996-5f88-4f7d-9525-98a0caeb8fcd None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:22.105355 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:22.224510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81571996-5f88-4f7d-9525-98a0caeb8fcd None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:22.268102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576a6c5-58ea-4b0b-be17-9da3af65b7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:22.268739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576a6c5-58ea-4b0b-be17-9da3af65b7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:22.269018 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576a6c5-58ea-4b0b-be17-9da3af65b7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:22.269293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576a6c5-58ea-4b0b-be17-9da3af65b7e3 None admin] PATH_INFO: `/v3/users/fc2bbe3ff1b049a7af86aaa72244780c` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:22.270445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576a6c5-58ea-4b0b-be17-9da3af65b7e3 None admin] RBAC: Authorizing `identity:get_user(user_id=fc2bbe3ff1b049a7af86aaa72244780c)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:22.271896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d576a6c5-58ea-4b0b-be17-9da3af65b7e3 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:22.273469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 73/146] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:22 2022] GET /identity/v3/users/fc2bbe3ff1b049a7af86aaa72244780c => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:22.279146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c67869-3d64-4f79-95ab-15e9ceb5fb59 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:22.428718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c67869-3d64-4f79-95ab-15e9ceb5fb59 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:22.431179 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:22.469369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86824e70-2ce1-43cf-b488-09fd925f1763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:22.470086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86824e70-2ce1-43cf-b488-09fd925f1763 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:22.471087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86824e70-2ce1-43cf-b488-09fd925f1763 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:22.471752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86824e70-2ce1-43cf-b488-09fd925f1763 None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:22.473026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86824e70-2ce1-43cf-b488-09fd925f1763 None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:22.475068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86824e70-2ce1-43cf-b488-09fd925f1763 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:22.476680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 74/147] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:22 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:22.482432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576a6c5-58ea-4b0b-be17-9da3af65b7e3 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:22.582760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:22.600802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576a6c5-58ea-4b0b-be17-9da3af65b7e3 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:22.645520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072c009-5253-4017-8b7a-0239a69e138b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:22.646201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072c009-5253-4017-8b7a-0239a69e138b None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:22.646469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072c009-5253-4017-8b7a-0239a69e138b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:22.646732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072c009-5253-4017-8b7a-0239a69e138b None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e/users/fc2bbe3ff1b049a7af86aaa72244780c/roles/c8fdda710e4f416380f2e8d61fa5b71f` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:22.648664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072c009-5253-4017-8b7a-0239a69e138b None admin] RBAC: Authorizing `identity:create_grant(project_id=703ed0f0d1ff4514b6dc0e857cbf989e, user_id=fc2bbe3ff1b049a7af86aaa72244780c, role_id=c8fdda710e4f416380f2e8d61fa5b71f)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:22.650109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072c009-5253-4017-8b7a-0239a69e138b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:22.780871 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 74/148] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:23:22 2022] PUT /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e/users/fc2bbe3ff1b049a7af86aaa72244780c/roles/c8fdda710e4f416380f2e8d61fa5b71f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:23:23.427962 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad143d-221f-4fac-8666-0dc70475a8f2 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:23.428807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad143d-221f-4fac-8666-0dc70475a8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:23.429226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad143d-221f-4fac-8666-0dc70475a8f2 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:23.431031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 75/149] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:23.437484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff59a6-d098-4235-be8d-571de187963a None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:23.437990 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff59a6-d098-4235-be8d-571de187963a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:23.438398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff59a6-d098-4235-be8d-571de187963a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:23.467784 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-03ff59a6-d098-4235-be8d-571de187963a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:23.468959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-03ff59a6-d098-4235-be8d-571de187963a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:23.476162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77525e56-c1a8-4018-8c50-34bd73d34229 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:23.476439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77525e56-c1a8-4018-8c50-34bd73d34229 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:23.476710 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77525e56-c1a8-4018-8c50-34bd73d34229 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:23.478349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 76/150] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:23.488376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d969e-56ed-417a-849e-203e784ec401 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:23.488655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d969e-56ed-417a-849e-203e784ec401 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:23.488913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d969e-56ed-417a-849e-203e784ec401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:23.518371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-ba6d969e-56ed-417a-849e-203e784ec401 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:23.519554 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ba6d969e-56ed-417a-849e-203e784ec401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:23.636470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 75/151] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:23.668155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 77/152] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:24.232314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff59a6-d098-4235-be8d-571de187963a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:24.249995 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d969e-56ed-417a-849e-203e784ec401 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:24.405684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:24.457954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d969e-56ed-417a-849e-203e784ec401 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:24.459461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:24.460003 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff59a6-d098-4235-be8d-571de187963a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:24.527174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd4b53-1cb0-47b6-9507-432ba161b0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:24.527350 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb743c-761f-4e23-af7b-093ec03acc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:24.527737 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd4b53-1cb0-47b6-9507-432ba161b0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:24.527996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb743c-761f-4e23-af7b-093ec03acc49 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:24.528094 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd4b53-1cb0-47b6-9507-432ba161b0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:24.528269 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb743c-761f-4e23-af7b-093ec03acc49 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:24.528361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd4b53-1cb0-47b6-9507-432ba161b0f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:24.528535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb743c-761f-4e23-af7b-093ec03acc49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:24.558794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd4b53-1cb0-47b6-9507-432ba161b0f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-75fd4b53-1cb0-47b6-9507-432ba161b0f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:24.564314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:24.565170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:24.565955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:24.565955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:24.565955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:24.565955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:24.565955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:24.565955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:24.565955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:24.565955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:24.565955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:24.565955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:24.565955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:24.565955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:24.566546 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb743c-761f-4e23-af7b-093ec03acc49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:24.566614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 76/153] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:24.566614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90614338-6fba-402f-a207-0e9acbac1da0 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:24.566614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90614338-6fba-402f-a207-0e9acbac1da0 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:24.567286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90614338-6fba-402f-a207-0e9acbac1da0 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:24.568340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 77/154] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:24.573149 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90614338-6fba-402f-a207-0e9acbac1da0 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-c0eb743c-761f-4e23-af7b-093ec03acc49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:24.581887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:24.584085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 78/155] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:23:24.586047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3c0c4-686a-4f49-9619-5e5e530a997a None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:24.592940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3c0c4-686a-4f49-9619-5e5e530a997a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:24.592940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3c0c4-686a-4f49-9619-5e5e530a997a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:24.622839 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-1de3c0c4-686a-4f49-9619-5e5e530a997a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:24.624419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-1de3c0c4-686a-4f49-9619-5e5e530a997a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:24.759162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 79/156] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:24.760383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3c0c4-686a-4f49-9619-5e5e530a997a None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:24.795623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90614338-6fba-402f-a207-0e9acbac1da0 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:24.858698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca78c67-5a9c-465b-8683-38ed70c546c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:24.859316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca78c67-5a9c-465b-8683-38ed70c546c2 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:24.859585 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca78c67-5a9c-465b-8683-38ed70c546c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:24.859850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca78c67-5a9c-465b-8683-38ed70c546c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:24.860686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca78c67-5a9c-465b-8683-38ed70c546c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:24.861403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca78c67-5a9c-465b-8683-38ed70c546c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:24.862525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca78c67-5a9c-465b-8683-38ed70c546c2 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:24.864458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:24.881586 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 78/157] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:24.882952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84b25c-f69b-49e9-aad9-00ce57149a52 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:24.883232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84b25c-f69b-49e9-aad9-00ce57149a52 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:24.883495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84b25c-f69b-49e9-aad9-00ce57149a52 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:24.886644 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 79/158] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:24.891954 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84b25c-f69b-49e9-aad9-00ce57149a52 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:24.936498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:24.956920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3c0c4-686a-4f49-9619-5e5e530a997a None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:25.033346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1f671-3ddb-456a-890c-9b10eecf4958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:25.033986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1f671-3ddb-456a-890c-9b10eecf4958 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:25.034256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1f671-3ddb-456a-890c-9b10eecf4958 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:25.034514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1f671-3ddb-456a-890c-9b10eecf4958 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:25.035173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1f671-3ddb-456a-890c-9b10eecf4958 None admin] RBAC: Adding query filter params (name=member) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:25.035464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1f671-3ddb-456a-890c-9b10eecf4958 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:25.036826 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1f671-3ddb-456a-890c-9b10eecf4958 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:25.066279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 80/159] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:25.067514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1eedb-02e9-42bc-9ee6-5e675717f17d None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:25.067796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1eedb-02e9-42bc-9ee6-5e675717f17d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:25.068063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae1eedb-02e9-42bc-9ee6-5e675717f17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:25.106881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-3ae1eedb-02e9-42bc-9ee6-5e675717f17d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:25.108049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3ae1eedb-02e9-42bc-9ee6-5e675717f17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:25.144697 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84b25c-f69b-49e9-aad9-00ce57149a52 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:25.212652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40814297-06e5-4db1-85aa-7fd5e26e8943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:25.213682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40814297-06e5-4db1-85aa-7fd5e26e8943 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:25.214181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40814297-06e5-4db1-85aa-7fd5e26e8943 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:25.214753 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40814297-06e5-4db1-85aa-7fd5e26e8943 None admin] PATH_INFO: `/v3/users/fd426537f3dc4593941437dc64e7c961` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:25.216078 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40814297-06e5-4db1-85aa-7fd5e26e8943 None admin] RBAC: Authorizing `identity:get_user(user_id=fd426537f3dc4593941437dc64e7c961)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:25.221844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40814297-06e5-4db1-85aa-7fd5e26e8943 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:25.223831 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 80/160] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:24 2022] GET /identity/v3/users/fd426537f3dc4593941437dc64e7c961 => generated 275 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:25.224789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40814297-06e5-4db1-85aa-7fd5e26e8943 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:25.239992 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 81/161] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:25.240875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1eedb-02e9-42bc-9ee6-5e675717f17d None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:25.345440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:25.413105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:25.417858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40814297-06e5-4db1-85aa-7fd5e26e8943 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:25.441106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae1eedb-02e9-42bc-9ee6-5e675717f17d None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:25.477556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68f5dc-1e44-4a19-8c53-5f56606903c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:25.479031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68f5dc-1e44-4a19-8c53-5f56606903c0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:25.479538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68f5dc-1e44-4a19-8c53-5f56606903c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:25.480011 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68f5dc-1e44-4a19-8c53-5f56606903c0 None admin] PATH_INFO: `/v3/users/fd426537f3dc4593941437dc64e7c961` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:25.481382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68f5dc-1e44-4a19-8c53-5f56606903c0 None admin] RBAC: Authorizing `identity:get_user(user_id=fd426537f3dc4593941437dc64e7c961)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:25.483159 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68f5dc-1e44-4a19-8c53-5f56606903c0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:25.484955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 81/162] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:25 2022] GET /identity/v3/users/fd426537f3dc4593941437dc64e7c961 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:25.491215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68f5dc-1e44-4a19-8c53-5f56606903c0 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:25.511431 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb845fa7-0940-44be-a051-b03756d18969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:25.517932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb845fa7-0940-44be-a051-b03756d18969 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:25.518367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb845fa7-0940-44be-a051-b03756d18969 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:25.518783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb845fa7-0940-44be-a051-b03756d18969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:25.653746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb845fa7-0940-44be-a051-b03756d18969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:25.655490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb845fa7-0940-44be-a051-b03756d18969 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:25.692765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68f5dc-1e44-4a19-8c53-5f56606903c0 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:25.775893 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:25.785330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc107ae-7c6e-445c-aa0f-7433dfda8e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:25.786326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc107ae-7c6e-445c-aa0f-7433dfda8e46 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:25.786819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc107ae-7c6e-445c-aa0f-7433dfda8e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:25.787223 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc107ae-7c6e-445c-aa0f-7433dfda8e46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:25.833856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:25.857745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 82/163] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:25.864066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb845fa7-0940-44be-a051-b03756d18969 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:25.942098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc107ae-7c6e-445c-aa0f-7433dfda8e46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:25.943559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc107ae-7c6e-445c-aa0f-7433dfda8e46 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:26.063288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb845fa7-0940-44be-a051-b03756d18969 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:26.080453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 82/164] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 05:23:26.081557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06797aa-3ea1-484c-8f60-1fbb0fc54f4d None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:26.081848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06797aa-3ea1-484c-8f60-1fbb0fc54f4d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:26.082454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06797aa-3ea1-484c-8f60-1fbb0fc54f4d None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:26.083694 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 83/165] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:26.084305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06797aa-3ea1-484c-8f60-1fbb0fc54f4d None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:26.118252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:26.148675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04789465-d96a-4f03-b1b1-8b28ebbac06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:26.149327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04789465-d96a-4f03-b1b1-8b28ebbac06e None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:26.149607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04789465-d96a-4f03-b1b1-8b28ebbac06e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:26.149902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04789465-d96a-4f03-b1b1-8b28ebbac06e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:26.167968 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04789465-d96a-4f03-b1b1-8b28ebbac06e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-04789465-d96a-4f03-b1b1-8b28ebbac06e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:26.179719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:26.180614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:26.181404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:26.181404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:26.181404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:26.181404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:26.181404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:26.181404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:26.181404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:26.181404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:26.181404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:26.181404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:26.181404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:26.181404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:26.181404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 83/166] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:23:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:26.182274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04789465-d96a-4f03-b1b1-8b28ebbac06e None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:26.195428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06797aa-3ea1-484c-8f60-1fbb0fc54f4d None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:26.232876 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec4d8c-4eb6-42ab-b6f9-a5dcd2be7a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:26.233483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec4d8c-4eb6-42ab-b6f9-a5dcd2be7a11 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:26.233751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec4d8c-4eb6-42ab-b6f9-a5dcd2be7a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:26.234138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec4d8c-4eb6-42ab-b6f9-a5dcd2be7a11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:26.247284 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec4d8c-4eb6-42ab-b6f9-a5dcd2be7a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-c2ec4d8c-4eb6-42ab-b6f9-a5dcd2be7a11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:26.258725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:26.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:26.260493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:26.260493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:26.260493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:26.260493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:26.260493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:26.260493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:26.260493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:26.260493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:26.260493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:26.260493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:26.260493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:26.260493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:26.260493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 84/167] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:23:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:26.261235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedb463-73f8-452c-b009-b4eba3729485 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:26.261315 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedb463-73f8-452c-b009-b4eba3729485 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:26.262190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dedb463-73f8-452c-b009-b4eba3729485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:26.271844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:26.286501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-2dedb463-73f8-452c-b009-b4eba3729485 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:26.287691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2dedb463-73f8-452c-b009-b4eba3729485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:26.374822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04789465-d96a-4f03-b1b1-8b28ebbac06e None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:26.390666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 85/168] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:26.392615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dedb463-73f8-452c-b009-b4eba3729485 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:26.423594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:26.454654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd030e5-46c0-483b-be06-947d01e3598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:26.455210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd030e5-46c0-483b-be06-947d01e3598f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:26.455481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd030e5-46c0-483b-be06-947d01e3598f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:26.455744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd030e5-46c0-483b-be06-947d01e3598f None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:26.459160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd030e5-46c0-483b-be06-947d01e3598f None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:26.460654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd030e5-46c0-483b-be06-947d01e3598f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:26.466896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 84/169] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:26 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:26.473182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd030e5-46c0-483b-be06-947d01e3598f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:26.538544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dedb463-73f8-452c-b009-b4eba3729485 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:26.576630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770569fd-885f-47f1-a6d9-530c84dee052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:26.577540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770569fd-885f-47f1-a6d9-530c84dee052 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:26.578020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770569fd-885f-47f1-a6d9-530c84dee052 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:26.578437 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770569fd-885f-47f1-a6d9-530c84dee052 None admin] PATH_INFO: `/v3/projects/01f11c3472b641ea82fb9a5f60117973` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:26.580299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770569fd-885f-47f1-a6d9-530c84dee052 None admin] RBAC: Authorizing `identity:get_project(project_id=01f11c3472b641ea82fb9a5f60117973)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:26.582245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770569fd-885f-47f1-a6d9-530c84dee052 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:26.584485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 86/170] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:26 2022] GET /identity/v3/projects/01f11c3472b641ea82fb9a5f60117973 => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:26.585496 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770569fd-885f-47f1-a6d9-530c84dee052 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:26.686221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:26.698380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770569fd-885f-47f1-a6d9-530c84dee052 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:26.699367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd030e5-46c0-483b-be06-947d01e3598f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:26.725496 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:26.739056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfef0fe-25a4-4e40-8b27-340bf3bb86e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:26.740747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfef0fe-25a4-4e40-8b27-340bf3bb86e6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:26.740747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfef0fe-25a4-4e40-8b27-340bf3bb86e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:26.740747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfef0fe-25a4-4e40-8b27-340bf3bb86e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:26.741013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfef0fe-25a4-4e40-8b27-340bf3bb86e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:26.741466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfef0fe-25a4-4e40-8b27-340bf3bb86e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:26.752546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfef0fe-25a4-4e40-8b27-340bf3bb86e6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:26.768042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 87/171] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:23:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:26.769118 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfef0fe-25a4-4e40-8b27-340bf3bb86e6 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:26.772829 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35316fc9-b2e1-4dee-a0a2-ded718c2f5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:26.773516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35316fc9-b2e1-4dee-a0a2-ded718c2f5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:26.774086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35316fc9-b2e1-4dee-a0a2-ded718c2f5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:26.774571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35316fc9-b2e1-4dee-a0a2-ded718c2f5a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:26.776763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35316fc9-b2e1-4dee-a0a2-ded718c2f5a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:26.777324 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35316fc9-b2e1-4dee-a0a2-ded718c2f5a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:26.779120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35316fc9-b2e1-4dee-a0a2-ded718c2f5a5 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:26.790242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 85/172] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:26.791444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35316fc9-b2e1-4dee-a0a2-ded718c2f5a5 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:26.890400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfef0fe-25a4-4e40-8b27-340bf3bb86e6 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:26.912779 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35316fc9-b2e1-4dee-a0a2-ded718c2f5a5 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:26.927736 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea05e57-4a31-4d29-b388-8141df8c18b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:26.928333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea05e57-4a31-4d29-b388-8141df8c18b5 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:26.928602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea05e57-4a31-4d29-b388-8141df8c18b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:26.928866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea05e57-4a31-4d29-b388-8141df8c18b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:26.929795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea05e57-4a31-4d29-b388-8141df8c18b5 None admin] RBAC: Adding query filter params (role.id=413cc5f8baab440bab827a3547b65f33, scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e, user.id=fd426537f3dc4593941437dc64e7c961) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:26.929936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea05e57-4a31-4d29-b388-8141df8c18b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:26.931301 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea05e57-4a31-4d29-b388-8141df8c18b5 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:26.936850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 88/173] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:26 2022] GET /identity/v3/role_assignments?user.id=fd426537f3dc4593941437dc64e7c961&scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e&role.id=413cc5f8baab440bab827a3547b65f33 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:26.938059 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f83a6b-d568-4dd1-a77c-e396b64121fc None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:26.938417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f83a6b-d568-4dd1-a77c-e396b64121fc None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:26.939181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f83a6b-d568-4dd1-a77c-e396b64121fc None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:26.940363 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 89/174] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:26.943954 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87478c12-431a-4cde-b6c6-00cff294fb23 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:26.943954 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87478c12-431a-4cde-b6c6-00cff294fb23 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:26.943954 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87478c12-431a-4cde-b6c6-00cff294fb23 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:26.943954 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 90/175] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:26.944278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87478c12-431a-4cde-b6c6-00cff294fb23 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:26.959885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466da794-3cfb-4191-b101-28596755325d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:26.960600 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466da794-3cfb-4191-b101-28596755325d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:26.960878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466da794-3cfb-4191-b101-28596755325d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:26.961259 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466da794-3cfb-4191-b101-28596755325d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:26.962063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466da794-3cfb-4191-b101-28596755325d None admin] RBAC: Adding query filter params (role.id=413cc5f8baab440bab827a3547b65f33, scope.project.id=01f11c3472b641ea82fb9a5f60117973, user.id=fd426537f3dc4593941437dc64e7c961) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:26.962371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466da794-3cfb-4191-b101-28596755325d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:26.963877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466da794-3cfb-4191-b101-28596755325d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:26.970793 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 86/176] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:26 2022] GET /identity/v3/role_assignments?user.id=fd426537f3dc4593941437dc64e7c961&scope.project.id=01f11c3472b641ea82fb9a5f60117973&role.id=413cc5f8baab440bab827a3547b65f33 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:26.972904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:26.978959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466da794-3cfb-4191-b101-28596755325d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:27.044402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:27.085801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87478c12-431a-4cde-b6c6-00cff294fb23 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:27.136659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466da794-3cfb-4191-b101-28596755325d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:27.137095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537df64-64cf-4633-8ef5-5796aab5beec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:27.137743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537df64-64cf-4633-8ef5-5796aab5beec None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:27.138058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537df64-64cf-4633-8ef5-5796aab5beec None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:27.138320 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537df64-64cf-4633-8ef5-5796aab5beec None admin] PATH_INFO: `/v3/users/fd426537f3dc4593941437dc64e7c961` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:27.140219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537df64-64cf-4633-8ef5-5796aab5beec None admin] RBAC: Authorizing `identity:get_user(user_id=fd426537f3dc4593941437dc64e7c961)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:27.142208 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537df64-64cf-4633-8ef5-5796aab5beec None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:27.144072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 91/177] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:26 2022] GET /identity/v3/users/fd426537f3dc4593941437dc64e7c961 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:27.144685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537df64-64cf-4633-8ef5-5796aab5beec None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:27.186248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d1df6-1147-4044-8954-b7b20d74bcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:27.187652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d1df6-1147-4044-8954-b7b20d74bcee None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:27.188100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d1df6-1147-4044-8954-b7b20d74bcee None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:27.189349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d1df6-1147-4044-8954-b7b20d74bcee None admin] PATH_INFO: `/v3/users/2ebd28e621914177981b750fe6a7566b` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:27.190667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d1df6-1147-4044-8954-b7b20d74bcee None admin] RBAC: Authorizing `identity:get_user(user_id=2ebd28e621914177981b750fe6a7566b)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:27.192136 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d1df6-1147-4044-8954-b7b20d74bcee None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:27.194041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 87/178] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:26 2022] GET /identity/v3/users/2ebd28e621914177981b750fe6a7566b => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:27.197873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbd362-505c-4b28-9243-7dc7b48da62f None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:27.197873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbd362-505c-4b28-9243-7dc7b48da62f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:27.197873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbd362-505c-4b28-9243-7dc7b48da62f None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:27.197873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 88/179] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:27.198997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deae215-2b3e-4b0f-b077-0eb25eeeb598 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:27.199335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deae215-2b3e-4b0f-b077-0eb25eeeb598 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:27.199608 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deae215-2b3e-4b0f-b077-0eb25eeeb598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:27.220237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-5deae215-2b3e-4b0f-b077-0eb25eeeb598 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:27.221444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5deae215-2b3e-4b0f-b077-0eb25eeeb598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:27.259080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:27.275413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537df64-64cf-4633-8ef5-5796aab5beec None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:27.302455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 89/180] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:27.303916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f16bc9-cda3-4a9b-ae1a-b00da389398c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:27.304260 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f16bc9-cda3-4a9b-ae1a-b00da389398c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:27.304597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f16bc9-cda3-4a9b-ae1a-b00da389398c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:27.310521 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5faca7-ad58-4678-b5e8-1d6a12efa07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:27.311138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5faca7-ad58-4678-b5e8-1d6a12efa07a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:27.311408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5faca7-ad58-4678-b5e8-1d6a12efa07a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:27.311669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5faca7-ad58-4678-b5e8-1d6a12efa07a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:27.322206 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-64f16bc9-cda3-4a9b-ae1a-b00da389398c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:27.323111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5faca7-ad58-4678-b5e8-1d6a12efa07a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:27.323464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-64f16bc9-cda3-4a9b-ae1a-b00da389398c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-df5faca7-ad58-4678-b5e8-1d6a12efa07a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:27.328306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:27.329515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:27.330413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:27.330413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:27.330413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:27.330413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:27.330413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:27.330413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:27.330413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:27.330413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:27.330413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:27.330413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:27.330413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:27.330413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:27.331502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 92/181] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:23:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:27.332297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5faca7-ad58-4678-b5e8-1d6a12efa07a None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:27.354680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:27.400592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 90/182] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:27.400592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7be505-3e88-4d36-a079-ba0ceeefaa5a None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:27.400592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7be505-3e88-4d36-a079-ba0ceeefaa5a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:27.400592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7be505-3e88-4d36-a079-ba0ceeefaa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:27.413895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-9a7be505-3e88-4d36-a079-ba0ceeefaa5a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:27.415343 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-9a7be505-3e88-4d36-a079-ba0ceeefaa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:27.448872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5faca7-ad58-4678-b5e8-1d6a12efa07a None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:27.487382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b1602-1370-452e-bb04-bba6798977ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:27.488253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b1602-1370-452e-bb04-bba6798977ae None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:27.488642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b1602-1370-452e-bb04-bba6798977ae None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:27.488979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b1602-1370-452e-bb04-bba6798977ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:27.500485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 91/183] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:27.501527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7be505-3e88-4d36-a079-ba0ceeefaa5a None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:27.526092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:27.621772 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b1602-1370-452e-bb04-bba6798977ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:27.623270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b1602-1370-452e-bb04-bba6798977ae None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:27.643902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7be505-3e88-4d36-a079-ba0ceeefaa5a None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:27.653128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:27.693037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a5b5e-d5ca-4292-b911-28643e60e903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:27.694071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a5b5e-d5ca-4292-b911-28643e60e903 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:27.694639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a5b5e-d5ca-4292-b911-28643e60e903 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:27.694910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a5b5e-d5ca-4292-b911-28643e60e903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:27.741162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 93/184] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:27.741162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b1602-1370-452e-bb04-bba6798977ae None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:27.770728 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a5b5e-d5ca-4292-b911-28643e60e903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:27.772291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a5b5e-d5ca-4292-b911-28643e60e903 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:27.786532 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:27.841688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b1602-1370-452e-bb04-bba6798977ae None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:27.875897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee8a59-8e2d-44e5-a90c-443016f3e429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:27.876574 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee8a59-8e2d-44e5-a90c-443016f3e429 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:27.876846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee8a59-8e2d-44e5-a90c-443016f3e429 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:27.877213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee8a59-8e2d-44e5-a90c-443016f3e429 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:27.877323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:27.877980 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee8a59-8e2d-44e5-a90c-443016f3e429 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:27.878105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee8a59-8e2d-44e5-a90c-443016f3e429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:27.879435 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee8a59-8e2d-44e5-a90c-443016f3e429 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:27.883642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 92/185] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:27.887269 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a5b5e-d5ca-4292-b911-28643e60e903 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:27.887621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 94/186] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:27.888410 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee8a59-8e2d-44e5-a90c-443016f3e429 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:27.988129 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a5b5e-d5ca-4292-b911-28643e60e903 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:27.988333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee8a59-8e2d-44e5-a90c-443016f3e429 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:27.994951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:28.022614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc781a4-d418-4370-bdb3-0ecbca34b35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:28.022822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc86f1-756b-4b38-88a8-bca3bda289a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:28.023237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc781a4-d418-4370-bdb3-0ecbca34b35a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:28.023337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc86f1-756b-4b38-88a8-bca3bda289a3 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:28.023508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc781a4-d418-4370-bdb3-0ecbca34b35a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:28.023594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc86f1-756b-4b38-88a8-bca3bda289a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:28.023770 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc781a4-d418-4370-bdb3-0ecbca34b35a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:28.023856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc86f1-756b-4b38-88a8-bca3bda289a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:28.034214 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc781a4-d418-4370-bdb3-0ecbca34b35a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:28.035039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc86f1-756b-4b38-88a8-bca3bda289a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-1cc781a4-d418-4370-bdb3-0ecbca34b35a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:28.039161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:28.040115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:28.040896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:28.040896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:28.040896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:28.040896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:28.040896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:28.040896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:28.040896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:28.040896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:28.040896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:28.040896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:28.040896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:28.040896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-36dc86f1-756b-4b38-88a8-bca3bda289a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:28.042169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:28.042986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:28.043768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:28.043768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:28.043768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:28.043768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:28.043768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:28.043768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:28.043768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:28.043768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:28.043768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:28.043768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:28.043768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:28.043768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:28.044364 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 93/187] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:23:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:28.044364 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc781a4-d418-4370-bdb3-0ecbca34b35a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:28.044488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 95/188] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:28.044488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc86f1-756b-4b38-88a8-bca3bda289a3 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:28.097700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:28.138790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc781a4-d418-4370-bdb3-0ecbca34b35a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:28.143043 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc86f1-756b-4b38-88a8-bca3bda289a3 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:28.171476 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c8eb5-04b1-4cc7-a67b-29dd19238d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:28.172091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c8eb5-04b1-4cc7-a67b-29dd19238d74 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:28.172359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c8eb5-04b1-4cc7-a67b-29dd19238d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:28.172621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c8eb5-04b1-4cc7-a67b-29dd19238d74 None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:28.174353 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c8eb5-04b1-4cc7-a67b-29dd19238d74 None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:28.175787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c8eb5-04b1-4cc7-a67b-29dd19238d74 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:28.175935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2de10-f749-4db6-9dc9-5d3472d6083e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:28.176544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2de10-f749-4db6-9dc9-5d3472d6083e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:28.176813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2de10-f749-4db6-9dc9-5d3472d6083e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:28.177075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2de10-f749-4db6-9dc9-5d3472d6083e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:28.177391 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 94/189] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:28 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:28.178160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c8eb5-04b1-4cc7-a67b-29dd19238d74 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:28.178265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2de10-f749-4db6-9dc9-5d3472d6083e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:28.179687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2de10-f749-4db6-9dc9-5d3472d6083e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:28.181213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 96/190] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:28.182311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2de10-f749-4db6-9dc9-5d3472d6083e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:28.215640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:28.280299 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c8eb5-04b1-4cc7-a67b-29dd19238d74 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:28.280684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2de10-f749-4db6-9dc9-5d3472d6083e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:28.303598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:28.314282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c9876-fe1b-4d40-92d8-4017fc45439f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:28.316063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c9876-fe1b-4d40-92d8-4017fc45439f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:28.316063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c9876-fe1b-4d40-92d8-4017fc45439f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:28.316242 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce4ca6-70a7-48c7-badf-e62d5e2df4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:28.316328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c9876-fe1b-4d40-92d8-4017fc45439f None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:28.316400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce4ca6-70a7-48c7-badf-e62d5e2df4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:28.316400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce4ca6-70a7-48c7-badf-e62d5e2df4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:28.316530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c9876-fe1b-4d40-92d8-4017fc45439f None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:28.316614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afce4ca6-70a7-48c7-badf-e62d5e2df4f0 None admin] PATH_INFO: `/v3/users/4f1bb48567b146d4a0b3c1ee3c7d8e11` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:28.317910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c9876-fe1b-4d40-92d8-4017fc45439f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:28.319514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 95/191] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:28 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:28.320266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c9876-fe1b-4d40-92d8-4017fc45439f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:28.333675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce4ca6-70a7-48c7-badf-e62d5e2df4f0 None admin] RBAC: Authorizing `identity:get_user(user_id=4f1bb48567b146d4a0b3c1ee3c7d8e11)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:28.335171 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afce4ca6-70a7-48c7-badf-e62d5e2df4f0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:28.336820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 97/192] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:28 2022] GET /identity/v3/users/4f1bb48567b146d4a0b3c1ee3c7d8e11 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:28.337572 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce4ca6-70a7-48c7-badf-e62d5e2df4f0 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:28.417212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c9876-fe1b-4d40-92d8-4017fc45439f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:28.427510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:28.435681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afce4ca6-70a7-48c7-badf-e62d5e2df4f0 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:28.451113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01ce80-54a1-41b4-88e6-b7477b1d1227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:28.451766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01ce80-54a1-41b4-88e6-b7477b1d1227 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:28.452043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01ce80-54a1-41b4-88e6-b7477b1d1227 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:28.452310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01ce80-54a1-41b4-88e6-b7477b1d1227 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:28.452973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01ce80-54a1-41b4-88e6-b7477b1d1227 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:28.453303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01ce80-54a1-41b4-88e6-b7477b1d1227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:28.455642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01ce80-54a1-41b4-88e6-b7477b1d1227 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:28.463379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 96/193] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:23:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:28.464437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01ce80-54a1-41b4-88e6-b7477b1d1227 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:28.475321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefc2dd-352b-43c1-92f3-5f84722f1ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:28.475933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefc2dd-352b-43c1-92f3-5f84722f1ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:28.476507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefc2dd-352b-43c1-92f3-5f84722f1ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:28.476818 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefc2dd-352b-43c1-92f3-5f84722f1ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:28.477482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefc2dd-352b-43c1-92f3-5f84722f1ae5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:28.477782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefc2dd-352b-43c1-92f3-5f84722f1ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:28.479162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefc2dd-352b-43c1-92f3-5f84722f1ae5 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:28.487299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 98/194] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:28.488047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefc2dd-352b-43c1-92f3-5f84722f1ae5 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:28.520218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:28.564038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01ce80-54a1-41b4-88e6-b7477b1d1227 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:28.588963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefc2dd-352b-43c1-92f3-5f84722f1ae5 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:28.597400 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cbd202-6adf-46ab-9d70-7a11f7a03b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:28.598037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbd202-6adf-46ab-9d70-7a11f7a03b9d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:28.598305 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbd202-6adf-46ab-9d70-7a11f7a03b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:28.598568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbd202-6adf-46ab-9d70-7a11f7a03b9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:28.599289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cbd202-6adf-46ab-9d70-7a11f7a03b9d None admin] RBAC: Adding query filter params (role.id=93f699f03f0f41498415ab9533bdfcb4, scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e, user.id=fd426537f3dc4593941437dc64e7c961) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:28.599580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cbd202-6adf-46ab-9d70-7a11f7a03b9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:28.601219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cbd202-6adf-46ab-9d70-7a11f7a03b9d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:28.606944 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 97/195] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:28 2022] GET /identity/v3/role_assignments?user.id=fd426537f3dc4593941437dc64e7c961&scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e&role.id=93f699f03f0f41498415ab9533bdfcb4 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:28.607712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cbd202-6adf-46ab-9d70-7a11f7a03b9d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:28.627630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d89cb-4c1b-4b82-b38f-7589456a7555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:28.628357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d89cb-4c1b-4b82-b38f-7589456a7555 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:28.628633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d89cb-4c1b-4b82-b38f-7589456a7555 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:28.628905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d89cb-4c1b-4b82-b38f-7589456a7555 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:28.629633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d89cb-4c1b-4b82-b38f-7589456a7555 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:28.631246 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d89cb-4c1b-4b82-b38f-7589456a7555 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:28.718039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cbd202-6adf-46ab-9d70-7a11f7a03b9d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:28.726120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 99/196] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:23:28 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:28.726120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d89cb-4c1b-4b82-b38f-7589456a7555 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:28.749988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:28.756274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:28.781412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0db43-da0b-49d8-b5a3-73e40d8105f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:28.782108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0db43-da0b-49d8-b5a3-73e40d8105f6 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:28.782989 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0db43-da0b-49d8-b5a3-73e40d8105f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:28.782989 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0db43-da0b-49d8-b5a3-73e40d8105f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:28.783874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0db43-da0b-49d8-b5a3-73e40d8105f6 None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e, user.id=2ebd28e621914177981b750fe6a7566b) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:28.784185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0db43-da0b-49d8-b5a3-73e40d8105f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:28.786170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0db43-da0b-49d8-b5a3-73e40d8105f6 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:28.801926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 98/197] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:28 2022] GET /identity/v3/role_assignments?user.id=2ebd28e621914177981b750fe6a7566b&scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:28.801926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0db43-da0b-49d8-b5a3-73e40d8105f6 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:28.871330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d89cb-4c1b-4b82-b38f-7589456a7555 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:28.918907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33918770-9c28-40d7-9a54-491f3fb1d494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:28.919635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33918770-9c28-40d7-9a54-491f3fb1d494 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:28.920096 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33918770-9c28-40d7-9a54-491f3fb1d494 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:28.920361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33918770-9c28-40d7-9a54-491f3fb1d494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:28.945738 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0db43-da0b-49d8-b5a3-73e40d8105f6 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:28.998074 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbcbe0-2a0c-4612-8884-8ecf9ef3b0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:28.998724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbcbe0-2a0c-4612-8884-8ecf9ef3b0f3 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:29.000145 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbcbe0-2a0c-4612-8884-8ecf9ef3b0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:29.000443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbcbe0-2a0c-4612-8884-8ecf9ef3b0f3 None admin] PATH_INFO: `/v3/users/4f1bb48567b146d4a0b3c1ee3c7d8e11` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:29.002492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbcbe0-2a0c-4612-8884-8ecf9ef3b0f3 None admin] RBAC: Authorizing `identity:get_user(user_id=4f1bb48567b146d4a0b3c1ee3c7d8e11)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:29.004098 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbcbe0-2a0c-4612-8884-8ecf9ef3b0f3 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:29.005868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 99/198] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:28 2022] GET /identity/v3/users/4f1bb48567b146d4a0b3c1ee3c7d8e11 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:29.006652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbcbe0-2a0c-4612-8884-8ecf9ef3b0f3 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:29.034290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33918770-9c28-40d7-9a54-491f3fb1d494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:29.035756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33918770-9c28-40d7-9a54-491f3fb1d494 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:29.049799 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:29.076568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:29.148140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbcbe0-2a0c-4612-8884-8ecf9ef3b0f3 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:29.169184 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 100/199] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:29.169184 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33918770-9c28-40d7-9a54-491f3fb1d494 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:29.193590 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c06cd7-8992-4214-a77a-0f1a8a82becf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:29.194650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c06cd7-8992-4214-a77a-0f1a8a82becf None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:29.194917 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c06cd7-8992-4214-a77a-0f1a8a82becf None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:29.195189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c06cd7-8992-4214-a77a-0f1a8a82becf None admin] PATH_INFO: `/v3/users/fc2bbe3ff1b049a7af86aaa72244780c` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:29.196404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c06cd7-8992-4214-a77a-0f1a8a82becf None admin] RBAC: Authorizing `identity:get_user(user_id=fc2bbe3ff1b049a7af86aaa72244780c)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:29.197913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c06cd7-8992-4214-a77a-0f1a8a82becf None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:29.199526 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 100/200] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:29 2022] GET /identity/v3/users/fc2bbe3ff1b049a7af86aaa72244780c => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:29.204783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c06cd7-8992-4214-a77a-0f1a8a82becf None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:29.288919 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33918770-9c28-40d7-9a54-491f3fb1d494 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:29.300950 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:29.325451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c06cd7-8992-4214-a77a-0f1a8a82becf None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:29.327441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34b226-6674-457a-883a-8a8cc92ce645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:29.328054 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34b226-6674-457a-883a-8a8cc92ce645 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:29.328327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34b226-6674-457a-883a-8a8cc92ce645 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:29.328595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34b226-6674-457a-883a-8a8cc92ce645 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:29.346278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:29.368739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72ed28-ba8e-4cd1-9c03-2c6d301c3d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:29.369378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72ed28-ba8e-4cd1-9c03-2c6d301c3d2a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:29.369655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72ed28-ba8e-4cd1-9c03-2c6d301c3d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:29.369954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72ed28-ba8e-4cd1-9c03-2c6d301c3d2a None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:29.371604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72ed28-ba8e-4cd1-9c03-2c6d301c3d2a None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:29.373547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72ed28-ba8e-4cd1-9c03-2c6d301c3d2a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:29.377322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 101/201] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:29 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:29.378541 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72ed28-ba8e-4cd1-9c03-2c6d301c3d2a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:29.403126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34b226-6674-457a-883a-8a8cc92ce645 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:29.404511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34b226-6674-457a-883a-8a8cc92ce645 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:29.498240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72ed28-ba8e-4cd1-9c03-2c6d301c3d2a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:29.525971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 101/202] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:29.527124 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34b226-6674-457a-883a-8a8cc92ce645 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:29.535582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e969f67-3ad4-4a4d-b59d-85085e1e41c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:29.536275 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e969f67-3ad4-4a4d-b59d-85085e1e41c7 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:29.536774 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e969f67-3ad4-4a4d-b59d-85085e1e41c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:29.537119 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e969f67-3ad4-4a4d-b59d-85085e1e41c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:29.540750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:29.603030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:29.624694 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e969f67-3ad4-4a4d-b59d-85085e1e41c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:29.627252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e969f67-3ad4-4a4d-b59d-85085e1e41c7 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:29.646241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34b226-6674-457a-883a-8a8cc92ce645 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:29.727197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0100e9f-52ae-4fd5-bbcb-639d85a3c659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:29.727710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0100e9f-52ae-4fd5-bbcb-639d85a3c659 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:29.727989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0100e9f-52ae-4fd5-bbcb-639d85a3c659 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:29.728255 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0100e9f-52ae-4fd5-bbcb-639d85a3c659 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:29.729265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0100e9f-52ae-4fd5-bbcb-639d85a3c659 None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe, user.id=4f1bb48567b146d4a0b3c1ee3c7d8e11) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:29.729570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0100e9f-52ae-4fd5-bbcb-639d85a3c659 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:29.731339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0100e9f-52ae-4fd5-bbcb-639d85a3c659 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:29.739425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 102/203] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:29 2022] GET /identity/v3/role_assignments?user.id=4f1bb48567b146d4a0b3c1ee3c7d8e11&scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:29.740492 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0100e9f-52ae-4fd5-bbcb-639d85a3c659 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:29.790160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 102/204] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:29.808138 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e969f67-3ad4-4a4d-b59d-85085e1e41c7 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:29.862519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:29.895874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0100e9f-52ae-4fd5-bbcb-639d85a3c659 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:29.939220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286d5da-69aa-4bba-9c37-b8e21258360a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:29.940192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286d5da-69aa-4bba-9c37-b8e21258360a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:29.940642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286d5da-69aa-4bba-9c37-b8e21258360a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:29.941050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286d5da-69aa-4bba-9c37-b8e21258360a None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:29.942447 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286d5da-69aa-4bba-9c37-b8e21258360a None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:29.944416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286d5da-69aa-4bba-9c37-b8e21258360a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:29.946502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 103/205] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:29 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:29.955115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286d5da-69aa-4bba-9c37-b8e21258360a None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:30.018080 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:30.039912 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e969f67-3ad4-4a4d-b59d-85085e1e41c7 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:30.110321 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3837602-3754-42c7-af72-3028e42fe008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:30.112886 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3837602-3754-42c7-af72-3028e42fe008 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:30.115221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286d5da-69aa-4bba-9c37-b8e21258360a None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:30.115709 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3837602-3754-42c7-af72-3028e42fe008 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:30.116173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3837602-3754-42c7-af72-3028e42fe008 None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:30.118839 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3837602-3754-42c7-af72-3028e42fe008 None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:30.120510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3837602-3754-42c7-af72-3028e42fe008 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:30.123348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 103/206] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:29 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:30.144807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3837602-3754-42c7-af72-3028e42fe008 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:30.199405 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8018f5-1a5a-43b0-a022-fc252df3a1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:30.200017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8018f5-1a5a-43b0-a022-fc252df3a1e4 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:30.200289 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8018f5-1a5a-43b0-a022-fc252df3a1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:30.200552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8018f5-1a5a-43b0-a022-fc252df3a1e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:30.201276 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8018f5-1a5a-43b0-a022-fc252df3a1e4 None admin] RBAC: Adding query filter params (role.id=93f699f03f0f41498415ab9533bdfcb4, scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe, user.id=4f1bb48567b146d4a0b3c1ee3c7d8e11) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:30.201579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8018f5-1a5a-43b0-a022-fc252df3a1e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:30.203037 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8018f5-1a5a-43b0-a022-fc252df3a1e4 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:30.210515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 104/207] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:29 2022] GET /identity/v3/role_assignments?user.id=4f1bb48567b146d4a0b3c1ee3c7d8e11&scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe&role.id=93f699f03f0f41498415ab9533bdfcb4 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:30.332882 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3837602-3754-42c7-af72-3028e42fe008 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:30.379734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf154ffa-9bf1-4a39-b61a-7f27105d325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:30.380394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf154ffa-9bf1-4a39-b61a-7f27105d325c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:30.380662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf154ffa-9bf1-4a39-b61a-7f27105d325c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:30.380926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf154ffa-9bf1-4a39-b61a-7f27105d325c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:30.381698 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf154ffa-9bf1-4a39-b61a-7f27105d325c None admin] RBAC: Adding query filter params (role.id=c8fdda710e4f416380f2e8d61fa5b71f, scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e, user.id=fc2bbe3ff1b049a7af86aaa72244780c) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:30.382037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf154ffa-9bf1-4a39-b61a-7f27105d325c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:30.383452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf154ffa-9bf1-4a39-b61a-7f27105d325c None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:30.395905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 104/208] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:30 2022] GET /identity/v3/role_assignments?user.id=fc2bbe3ff1b049a7af86aaa72244780c&scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e&role.id=c8fdda710e4f416380f2e8d61fa5b71f => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:31.291860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86653fab-4327-4f2d-aabe-1ca5ab8ab8bf None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:31.292865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86653fab-4327-4f2d-aabe-1ca5ab8ab8bf None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:31.293346 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86653fab-4327-4f2d-aabe-1ca5ab8ab8bf None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:31.295067 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 105/209] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:31.301879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875774d-27a3-43a9-8f9b-2ae5d58e8c73 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:31.302884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875774d-27a3-43a9-8f9b-2ae5d58e8c73 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:31.303390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875774d-27a3-43a9-8f9b-2ae5d58e8c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:31.323614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-b875774d-27a3-43a9-8f9b-2ae5d58e8c73 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:31.325213 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b875774d-27a3-43a9-8f9b-2ae5d58e8c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:31.339990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:31.399180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 105/210] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:31.428388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc82a45-94e7-4691-abac-76e9efb705de None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:31.428966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc82a45-94e7-4691-abac-76e9efb705de None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:31.429458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc82a45-94e7-4691-abac-76e9efb705de None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:31.431040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 106/211] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:31.436121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aca3f6-8551-4afe-b1af-edb7e7751e89 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:31.436397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aca3f6-8551-4afe-b1af-edb7e7751e89 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:31.436666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aca3f6-8551-4afe-b1af-edb7e7751e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:31.454644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-85aca3f6-8551-4afe-b1af-edb7e7751e89 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:31.455851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-85aca3f6-8551-4afe-b1af-edb7e7751e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:31.521390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 106/212] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:31.676220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc82a45-94e7-4691-abac-76e9efb705de None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:31.686685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:31.784889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc82a45-94e7-4691-abac-76e9efb705de None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:31.799141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aca3f6-8551-4afe-b1af-edb7e7751e89 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:31.827104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be457862-b126-4543-a95b-dd86d672cf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:31.827868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be457862-b126-4543-a95b-dd86d672cf61 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:31.828145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be457862-b126-4543-a95b-dd86d672cf61 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:31.828409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be457862-b126-4543-a95b-dd86d672cf61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:31.847053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be457862-b126-4543-a95b-dd86d672cf61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-be457862-b126-4543-a95b-dd86d672cf61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:31.852756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:31.853638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:31.854473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:31.854473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:31.854473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:31.854473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:31.854473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:31.854473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:31.854473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:31.854473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:31.854473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:31.854473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:31.854473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:31.854473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:31.854473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 107/213] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:31.859927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be457862-b126-4543-a95b-dd86d672cf61 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:31.901716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:31.933324 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aca3f6-8551-4afe-b1af-edb7e7751e89 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:31.938651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:31.970985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99868f04-24e5-4864-9444-899482bac3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:31.971962 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99868f04-24e5-4864-9444-899482bac3ea None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:31.972380 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99868f04-24e5-4864-9444-899482bac3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:31.972791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99868f04-24e5-4864-9444-899482bac3ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:31.974219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99868f04-24e5-4864-9444-899482bac3ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:31.975892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99868f04-24e5-4864-9444-899482bac3ea None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:31.982146 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be457862-b126-4543-a95b-dd86d672cf61 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:31.982297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 107/214] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:31.984173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99868f04-24e5-4864-9444-899482bac3ea None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:32.024947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41e778-915c-4e32-9b31-c8d664eb0c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:32.025556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41e778-915c-4e32-9b31-c8d664eb0c72 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:32.025851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41e778-915c-4e32-9b31-c8d664eb0c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:32.026128 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41e778-915c-4e32-9b31-c8d664eb0c72 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:32.026790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41e778-915c-4e32-9b31-c8d664eb0c72 None admin] RBAC: Adding query filter params (name=member) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:32.027087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41e778-915c-4e32-9b31-c8d664eb0c72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:32.028468 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41e778-915c-4e32-9b31-c8d664eb0c72 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:32.041929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 108/215] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:32.044262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41e778-915c-4e32-9b31-c8d664eb0c72 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:32.114010 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99868f04-24e5-4864-9444-899482bac3ea None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:32.152702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41e778-915c-4e32-9b31-c8d664eb0c72 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:32.155718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26d3bd-0f5b-47eb-9bd5-e3dc7279d3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:32.156584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26d3bd-0f5b-47eb-9bd5-e3dc7279d3ac None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:32.157034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26d3bd-0f5b-47eb-9bd5-e3dc7279d3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:32.158011 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26d3bd-0f5b-47eb-9bd5-e3dc7279d3ac None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:32.158921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26d3bd-0f5b-47eb-9bd5-e3dc7279d3ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:32.160545 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26d3bd-0f5b-47eb-9bd5-e3dc7279d3ac None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:32.183453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:32.191720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29343241-1ecc-4997-8c51-08fc130afffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:32.192350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29343241-1ecc-4997-8c51-08fc130afffb None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:32.192623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29343241-1ecc-4997-8c51-08fc130afffb None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:32.192891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29343241-1ecc-4997-8c51-08fc130afffb None admin] PATH_INFO: `/v3/users/617786f5e72e4f638898771fb6ca88fc` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:32.206770 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29343241-1ecc-4997-8c51-08fc130afffb None admin] RBAC: Authorizing `identity:get_user(user_id=617786f5e72e4f638898771fb6ca88fc)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:32.208250 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29343241-1ecc-4997-8c51-08fc130afffb None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:32.210123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 109/216] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:32 2022] GET /identity/v3/users/617786f5e72e4f638898771fb6ca88fc => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:32.215399 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29343241-1ecc-4997-8c51-08fc130afffb None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:32.271983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 108/217] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:23:31 2022] POST /identity/v3/users => generated 297 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:32.271983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa36bb-c6ca-4866-a555-4276838e4340 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:32.271983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa36bb-c6ca-4866-a555-4276838e4340 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:32.271983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa36bb-c6ca-4866-a555-4276838e4340 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:32.271983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 109/218] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:32.280658 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f86253-cb9a-4e2a-9557-ff7681371087 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:32.280942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f86253-cb9a-4e2a-9557-ff7681371087 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:32.281215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f86253-cb9a-4e2a-9557-ff7681371087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:32.288311 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:32.307341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-02f86253-cb9a-4e2a-9557-ff7681371087 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:32.310180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-02f86253-cb9a-4e2a-9557-ff7681371087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:32.338471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29343241-1ecc-4997-8c51-08fc130afffb None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:32.382845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2092af-d538-4c23-a2ba-0390e40db6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:32.383622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2092af-d538-4c23-a2ba-0390e40db6ed None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:32.383998 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2092af-d538-4c23-a2ba-0390e40db6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:32.384366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2092af-d538-4c23-a2ba-0390e40db6ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:32.411284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 110/219] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:32.451218 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:32.465260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2092af-d538-4c23-a2ba-0390e40db6ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:32.467075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2092af-d538-4c23-a2ba-0390e40db6ed None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:32.575291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 110/220] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:32.582965 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f86253-cb9a-4e2a-9557-ff7681371087 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:32.659904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf583fa5-9ce6-42d8-85d8-276c31d74ec3 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:32.660235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf583fa5-9ce6-42d8-85d8-276c31d74ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:32.660447 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf583fa5-9ce6-42d8-85d8-276c31d74ec3 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:32.661789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 111/221] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:23:32.667633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c187f86-2f57-4387-a4ec-cc9fb10a8621 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:32.667911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c187f86-2f57-4387-a4ec-cc9fb10a8621 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:32.668184 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c187f86-2f57-4387-a4ec-cc9fb10a8621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:32.688684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-5c187f86-2f57-4387-a4ec-cc9fb10a8621 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:32.690008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5c187f86-2f57-4387-a4ec-cc9fb10a8621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:32.714833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f86253-cb9a-4e2a-9557-ff7681371087 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:32.747159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:32.752872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3ddc8-fff1-4f5f-9377-b14a5da06e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:32.753745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3ddc8-fff1-4f5f-9377-b14a5da06e95 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:32.754198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3ddc8-fff1-4f5f-9377-b14a5da06e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:32.765114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3ddc8-fff1-4f5f-9377-b14a5da06e95 None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:32.765114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3ddc8-fff1-4f5f-9377-b14a5da06e95 None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:32.765114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3ddc8-fff1-4f5f-9377-b14a5da06e95 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:32.765114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 111/222] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:32 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:32.765114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3ddc8-fff1-4f5f-9377-b14a5da06e95 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:32.773634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 112/223] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:32.774584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c187f86-2f57-4387-a4ec-cc9fb10a8621 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:32.804461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:32.912788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3ddc8-fff1-4f5f-9377-b14a5da06e95 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:32.956527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c24920-4057-418b-9fb5-c2783e7f008f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:32.958369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c24920-4057-418b-9fb5-c2783e7f008f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:32.959975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c24920-4057-418b-9fb5-c2783e7f008f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:32.960678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c24920-4057-418b-9fb5-c2783e7f008f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:32.962722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c187f86-2f57-4387-a4ec-cc9fb10a8621 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:32.984107 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c24920-4057-418b-9fb5-c2783e7f008f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-f0c24920-4057-418b-9fb5-c2783e7f008f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:32.997255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:32.999885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:33.000688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:33.000688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:33.000688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:33.000688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:33.000688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:33.000688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:33.000688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:33.000688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:33.000688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:33.000688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:33.000688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:33.000688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:33.003854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 112/224] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:23:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:33.009951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c24920-4057-418b-9fb5-c2783e7f008f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:33.037738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98056ba0-8956-42e2-8e1d-c8eaa0512cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:33.042453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98056ba0-8956-42e2-8e1d-c8eaa0512cbb None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:33.043913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98056ba0-8956-42e2-8e1d-c8eaa0512cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:33.044437 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98056ba0-8956-42e2-8e1d-c8eaa0512cbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:33.045557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98056ba0-8956-42e2-8e1d-c8eaa0512cbb None admin] RBAC: Adding query filter params (role.id=413cc5f8baab440bab827a3547b65f33, scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe, user.id=617786f5e72e4f638898771fb6ca88fc) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:33.047295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98056ba0-8956-42e2-8e1d-c8eaa0512cbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:33.049337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98056ba0-8956-42e2-8e1d-c8eaa0512cbb None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:33.065346 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 113/225] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:32 2022] GET /identity/v3/role_assignments?user.id=617786f5e72e4f638898771fb6ca88fc&scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe&role.id=413cc5f8baab440bab827a3547b65f33 => generated 259 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:33.097340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98056ba0-8956-42e2-8e1d-c8eaa0512cbb None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:33.110251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:33.192060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c24920-4057-418b-9fb5-c2783e7f008f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:33.237375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da0fac-c16e-4efe-b550-6eba07c64f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:33.238016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da0fac-c16e-4efe-b550-6eba07c64f09 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:33.238297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da0fac-c16e-4efe-b550-6eba07c64f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:33.238568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da0fac-c16e-4efe-b550-6eba07c64f09 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:33.239249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da0fac-c16e-4efe-b550-6eba07c64f09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:33.239552 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da0fac-c16e-4efe-b550-6eba07c64f09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:33.243065 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da0fac-c16e-4efe-b550-6eba07c64f09 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:33.252619 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:33.254038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 113/226] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:33.260638 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da0fac-c16e-4efe-b550-6eba07c64f09 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:33.289961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98056ba0-8956-42e2-8e1d-c8eaa0512cbb None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:33.363921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f9cad-8f5c-4473-9e08-c561eb4ef673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:33.364551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f9cad-8f5c-4473-9e08-c561eb4ef673 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:33.364828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f9cad-8f5c-4473-9e08-c561eb4ef673 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:33.365094 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f9cad-8f5c-4473-9e08-c561eb4ef673 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:33.383502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da0fac-c16e-4efe-b550-6eba07c64f09 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:33.396941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f9cad-8f5c-4473-9e08-c561eb4ef673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-fb4f9cad-8f5c-4473-9e08-c561eb4ef673 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:33.402328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:33.403212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:33.408642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:33.408642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:33.408642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:33.408642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:33.408642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:33.408642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:33.408642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:33.408642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:33.408642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:33.408642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:33.408642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:33.408642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:33.408642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 114/227] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:23:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:23:33.419271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:33.421544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f9cad-8f5c-4473-9e08-c561eb4ef673 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:33.432879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca62d4-7fca-46f9-b634-a3cda2e408a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:33.433806 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca62d4-7fca-46f9-b634-a3cda2e408a9 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:33.434323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca62d4-7fca-46f9-b634-a3cda2e408a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:33.435803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca62d4-7fca-46f9-b634-a3cda2e408a9 None admin] PATH_INFO: `/v3/users/4f1bb48567b146d4a0b3c1ee3c7d8e11` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:33.437214 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca62d4-7fca-46f9-b634-a3cda2e408a9 None admin] RBAC: Authorizing `identity:get_user(user_id=4f1bb48567b146d4a0b3c1ee3c7d8e11)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:33.439287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca62d4-7fca-46f9-b634-a3cda2e408a9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:33.441655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 114/228] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:33 2022] GET /identity/v3/users/4f1bb48567b146d4a0b3c1ee3c7d8e11 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:33.446737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca62d4-7fca-46f9-b634-a3cda2e408a9 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:33.570212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f9cad-8f5c-4473-9e08-c561eb4ef673 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:33.611451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca9b08-b67d-425c-8889-8fc1ea5ec8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:33.612443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca9b08-b67d-425c-8889-8fc1ea5ec8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:33.612922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca9b08-b67d-425c-8889-8fc1ea5ec8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:33.613387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca9b08-b67d-425c-8889-8fc1ea5ec8b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:33.614312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca9b08-b67d-425c-8889-8fc1ea5ec8b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:33.614831 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca9b08-b67d-425c-8889-8fc1ea5ec8b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:33.616497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ca9b08-b67d-425c-8889-8fc1ea5ec8b7 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:33.631647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 115/229] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:23:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:33.633168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca9b08-b67d-425c-8889-8fc1ea5ec8b7 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:33.636342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:33.683746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca62d4-7fca-46f9-b634-a3cda2e408a9 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:33.745385 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca9b08-b67d-425c-8889-8fc1ea5ec8b7 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:33.755976 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1209a-d149-425b-98a1-e7c7527df075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:33.756436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1209a-d149-425b-98a1-e7c7527df075 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:33.756709 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1209a-d149-425b-98a1-e7c7527df075 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:33.756982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1209a-d149-425b-98a1-e7c7527df075 None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:33.758475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1209a-d149-425b-98a1-e7c7527df075 None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:33.759925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1209a-d149-425b-98a1-e7c7527df075 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:33.761848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 115/230] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:33 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:33.771022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1209a-d149-425b-98a1-e7c7527df075 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:33.784933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911d356-fa67-4ab6-96ba-533b96315b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:33.785961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911d356-fa67-4ab6-96ba-533b96315b3f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:33.786442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911d356-fa67-4ab6-96ba-533b96315b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:33.786929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0911d356-fa67-4ab6-96ba-533b96315b3f None admin] PATH_INFO: `/v3/users/4f1bb48567b146d4a0b3c1ee3c7d8e11` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:33.788311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911d356-fa67-4ab6-96ba-533b96315b3f None admin] RBAC: Authorizing `identity:get_user(user_id=4f1bb48567b146d4a0b3c1ee3c7d8e11)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:33.790023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0911d356-fa67-4ab6-96ba-533b96315b3f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:33.796771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 116/231] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:33 2022] GET /identity/v3/users/4f1bb48567b146d4a0b3c1ee3c7d8e11 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:33.798703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911d356-fa67-4ab6-96ba-533b96315b3f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:33.881749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:33.916944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0911d356-fa67-4ab6-96ba-533b96315b3f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:33.920541 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:33.960501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d975bdb-d564-4961-9b89-412426772940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:33.961147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d975bdb-d564-4961-9b89-412426772940 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:33.961422 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d975bdb-d564-4961-9b89-412426772940 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:33.962226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d975bdb-d564-4961-9b89-412426772940 None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:33.963391 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d975bdb-d564-4961-9b89-412426772940 None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:33.965232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d975bdb-d564-4961-9b89-412426772940 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:33.970840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 117/232] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:33 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:33.973500 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d975bdb-d564-4961-9b89-412426772940 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:34.019826 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1209a-d149-425b-98a1-e7c7527df075 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:34.070469 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954b636-93e6-416d-98d5-1d71775c4b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:34.071175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954b636-93e6-416d-98d5-1d71775c4b18 None admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:34.071459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954b636-93e6-416d-98d5-1d71775c4b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:34.071779 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954b636-93e6-416d-98d5-1d71775c4b18 None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe/users/4f1bb48567b146d4a0b3c1ee3c7d8e11/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:34.073721 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954b636-93e6-416d-98d5-1d71775c4b18 None admin] RBAC: Authorizing `identity:create_grant(project_id=082a3596b7724c4f8f59b2687c2ca0fe, user_id=4f1bb48567b146d4a0b3c1ee3c7d8e11, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:34.075548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954b636-93e6-416d-98d5-1d71775c4b18 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:34.153133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d975bdb-d564-4961-9b89-412426772940 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:34.181125 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 116/233] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:23:33 2022] PUT /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe/users/4f1bb48567b146d4a0b3c1ee3c7d8e11/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:23:34.195327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3dba8-e419-4896-9270-13c83c38160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:34.196464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3dba8-e419-4896-9270-13c83c38160c None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:34.196943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3dba8-e419-4896-9270-13c83c38160c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:34.197459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3dba8-e419-4896-9270-13c83c38160c None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe/users/4f1bb48567b146d4a0b3c1ee3c7d8e11/roles/93f699f03f0f41498415ab9533bdfcb4` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:34.200106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3dba8-e419-4896-9270-13c83c38160c None admin] RBAC: Authorizing `identity:create_grant(project_id=082a3596b7724c4f8f59b2687c2ca0fe, user_id=4f1bb48567b146d4a0b3c1ee3c7d8e11, role_id=93f699f03f0f41498415ab9533bdfcb4)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:34.201914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3dba8-e419-4896-9270-13c83c38160c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:34.307099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:34.314850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 118/234] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:23:33 2022] PUT /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe/users/4f1bb48567b146d4a0b3c1ee3c7d8e11/roles/93f699f03f0f41498415ab9533bdfcb4 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:23:35.054250 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662b720-79ea-426a-94e3-06a1376aaba6 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:35.054749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662b720-79ea-426a-94e3-06a1376aaba6 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:35.054825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662b720-79ea-426a-94e3-06a1376aaba6 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:35.056413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 117/235] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:35.062175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3375660-43d4-4e3a-a334-042e1e085366 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:35.062596 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3375660-43d4-4e3a-a334-042e1e085366 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:35.062818 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3375660-43d4-4e3a-a334-042e1e085366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:35.081769 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-d3375660-43d4-4e3a-a334-042e1e085366 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:35.083152 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d3375660-43d4-4e3a-a334-042e1e085366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:35.159811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 119/236] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:35.186934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad94e4-dde9-4ea7-aaa2-02bb5d4190c1 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:35.187227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad94e4-dde9-4ea7-aaa2-02bb5d4190c1 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:35.187406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad94e4-dde9-4ea7-aaa2-02bb5d4190c1 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:35.189794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 118/237] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:35.194714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a108549-39d2-41af-ba31-8399bb0a5bbe None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:35.195001 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a108549-39d2-41af-ba31-8399bb0a5bbe None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:35.195276 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a108549-39d2-41af-ba31-8399bb0a5bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:35.224565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-2a108549-39d2-41af-ba31-8399bb0a5bbe None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:35.225756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2a108549-39d2-41af-ba31-8399bb0a5bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:35.319484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 120/238] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:35.536788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad94e4-dde9-4ea7-aaa2-02bb5d4190c1 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:35.636479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b2236-c590-4f4a-8ba0-9c872e6d128b None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:35.636898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b2236-c590-4f4a-8ba0-9c872e6d128b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:35.637168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b2236-c590-4f4a-8ba0-9c872e6d128b None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:35.639656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 121/239] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:35.645966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e531e-b566-40a1-8b8f-722f4cb33394 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:35.646246 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e531e-b566-40a1-8b8f-722f4cb33394 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:35.646517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e531e-b566-40a1-8b8f-722f4cb33394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:35.686443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-004e531e-b566-40a1-8b8f-722f4cb33394 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:35.688581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-004e531e-b566-40a1-8b8f-722f4cb33394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:35.693456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad94e4-dde9-4ea7-aaa2-02bb5d4190c1 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:35.717938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:35.739967 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca2513-ae6a-4456-908c-391c462d9594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:35.740602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca2513-ae6a-4456-908c-391c462d9594 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:35.740880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca2513-ae6a-4456-908c-391c462d9594 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:35.741151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca2513-ae6a-4456-908c-391c462d9594 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:35.742508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca2513-ae6a-4456-908c-391c462d9594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:35.744669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca2513-ae6a-4456-908c-391c462d9594 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:35.749968 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 119/240] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:35.749968 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca2513-ae6a-4456-908c-391c462d9594 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:35.780616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:35.793678 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 122/241] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:35.794871 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e531e-b566-40a1-8b8f-722f4cb33394 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:35.903939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca2513-ae6a-4456-908c-391c462d9594 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:35.943661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e531e-b566-40a1-8b8f-722f4cb33394 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:35.969100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa24eb-ae2d-400a-959b-ceee20772700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:35.969957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa24eb-ae2d-400a-959b-ceee20772700 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:35.970270 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa24eb-ae2d-400a-959b-ceee20772700 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:35.970535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa24eb-ae2d-400a-959b-ceee20772700 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:35.988997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa24eb-ae2d-400a-959b-ceee20772700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-d4fa24eb-ae2d-400a-959b-ceee20772700 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:35.995604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:35.996484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:35.997271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:35.997271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:35.997271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:35.997271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:35.997271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:35.997271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:35.997271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:35.997271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:35.997271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:35.997271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:35.997271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:35.997271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:35.997271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 120/242] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:36.003650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa24eb-ae2d-400a-959b-ceee20772700 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:36.011569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bd724-93fe-4556-9db3-24566e08d696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:36.012169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bd724-93fe-4556-9db3-24566e08d696 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:36.012499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bd724-93fe-4556-9db3-24566e08d696 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:36.012768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bd724-93fe-4556-9db3-24566e08d696 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:36.013424 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bd724-93fe-4556-9db3-24566e08d696 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:36.015318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bd724-93fe-4556-9db3-24566e08d696 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:36.073905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:36.099118 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 123/243] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:23:35 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:36.101776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bd724-93fe-4556-9db3-24566e08d696 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:36.148127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa24eb-ae2d-400a-959b-ceee20772700 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:36.167141 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:36.238807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bd724-93fe-4556-9db3-24566e08d696 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:36.242569 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638e24c-0b85-47a8-9353-930b6773f5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:36.243191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638e24c-0b85-47a8-9353-930b6773f5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:36.243475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638e24c-0b85-47a8-9353-930b6773f5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:36.243747 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9638e24c-0b85-47a8-9353-930b6773f5e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:36.244436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638e24c-0b85-47a8-9353-930b6773f5e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:36.244778 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638e24c-0b85-47a8-9353-930b6773f5e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:36.246196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9638e24c-0b85-47a8-9353-930b6773f5e8 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:36.265578 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 121/244] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:36.272128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638e24c-0b85-47a8-9353-930b6773f5e8 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:36.279336 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a903a-8c32-4b44-95ff-3f52edca7cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:36.279975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a903a-8c32-4b44-95ff-3f52edca7cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:36.280480 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a903a-8c32-4b44-95ff-3f52edca7cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:36.280757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a903a-8c32-4b44-95ff-3f52edca7cd8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:36.294623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a903a-8c32-4b44-95ff-3f52edca7cd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-ac9a903a-8c32-4b44-95ff-3f52edca7cd8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:36.301857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:36.304026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:36.306088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:36.306088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:36.306088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:36.306088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:36.306088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:36.306088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:36.306088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:36.306088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:36.306088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:36.306088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:36.306088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:36.306088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:36.306088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 124/245] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:36.313857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a903a-8c32-4b44-95ff-3f52edca7cd8 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:36.447035 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a903a-8c32-4b44-95ff-3f52edca7cd8 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:36.449373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:36.466675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9638e24c-0b85-47a8-9353-930b6773f5e8 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:36.477565 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:36.492407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232bd46-f05d-4f45-a4bb-d001b4a79d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:36.493019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232bd46-f05d-4f45-a4bb-d001b4a79d0b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:36.493292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232bd46-f05d-4f45-a4bb-d001b4a79d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:36.493559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232bd46-f05d-4f45-a4bb-d001b4a79d0b None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:36.494684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232bd46-f05d-4f45-a4bb-d001b4a79d0b None admin] RBAC: Adding query filter params (name=member) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:36.495051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232bd46-f05d-4f45-a4bb-d001b4a79d0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:36.496712 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232bd46-f05d-4f45-a4bb-d001b4a79d0b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:36.508596 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 125/246] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:36.513051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c055311-e0c2-4e77-bb7c-986c4f2a88ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:36.513968 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c055311-e0c2-4e77-bb7c-986c4f2a88ae None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:36.514349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c055311-e0c2-4e77-bb7c-986c4f2a88ae None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:36.514721 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c055311-e0c2-4e77-bb7c-986c4f2a88ae None admin] PATH_INFO: `/v3/users/712d0461c7c546ca87ccd5e92c2ae1be` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:36.515174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232bd46-f05d-4f45-a4bb-d001b4a79d0b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:36.531583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c055311-e0c2-4e77-bb7c-986c4f2a88ae None admin] RBAC: Authorizing `identity:get_user(user_id=712d0461c7c546ca87ccd5e92c2ae1be)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:36.534544 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c055311-e0c2-4e77-bb7c-986c4f2a88ae None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:36.541081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 122/247] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:36 2022] GET /identity/v3/users/712d0461c7c546ca87ccd5e92c2ae1be => generated 297 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:36.542484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c055311-e0c2-4e77-bb7c-986c4f2a88ae None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:36.642384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232bd46-f05d-4f45-a4bb-d001b4a79d0b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:36.688464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bade7d2-6c42-4a29-90a5-4db2a403494e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:36.689489 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bade7d2-6c42-4a29-90a5-4db2a403494e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:36.689844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bade7d2-6c42-4a29-90a5-4db2a403494e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:36.690238 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bade7d2-6c42-4a29-90a5-4db2a403494e None admin] PATH_INFO: `/v3/users/617786f5e72e4f638898771fb6ca88fc` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:36.691641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bade7d2-6c42-4a29-90a5-4db2a403494e None admin] RBAC: Authorizing `identity:get_user(user_id=617786f5e72e4f638898771fb6ca88fc)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:36.693059 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c055311-e0c2-4e77-bb7c-986c4f2a88ae None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:36.693331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bade7d2-6c42-4a29-90a5-4db2a403494e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:36.700356 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 126/248] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:36 2022] GET /identity/v3/users/617786f5e72e4f638898771fb6ca88fc => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:36.701911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bade7d2-6c42-4a29-90a5-4db2a403494e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:36.737620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaada350-3bd0-4dc0-80bb-9bf89ce7656d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:36.738494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaada350-3bd0-4dc0-80bb-9bf89ce7656d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:36.738908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaada350-3bd0-4dc0-80bb-9bf89ce7656d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:36.739309 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaada350-3bd0-4dc0-80bb-9bf89ce7656d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:36.755605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:36.776293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:36.811735 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaada350-3bd0-4dc0-80bb-9bf89ce7656d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:36.814124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaada350-3bd0-4dc0-80bb-9bf89ce7656d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:36.826602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bade7d2-6c42-4a29-90a5-4db2a403494e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:36.869476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aba775-e0f9-46bf-9fb1-110eae3efcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:36.870144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aba775-e0f9-46bf-9fb1-110eae3efcf8 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:36.870418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aba775-e0f9-46bf-9fb1-110eae3efcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:36.870681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aba775-e0f9-46bf-9fb1-110eae3efcf8 None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:36.871774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aba775-e0f9-46bf-9fb1-110eae3efcf8 None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:36.873208 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aba775-e0f9-46bf-9fb1-110eae3efcf8 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:36.879661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 127/249] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:36 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:36.879661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852aeb4-1786-45b8-ae58-e458f2f7634a None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:36.879661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852aeb4-1786-45b8-ae58-e458f2f7634a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:36.879661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2852aeb4-1786-45b8-ae58-e458f2f7634a None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:36.879661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 128/250] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:36.882774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852aeb4-1786-45b8-ae58-e458f2f7634a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:36.964931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 123/251] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 05:23:36.971713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc18268-f40a-40d2-bebf-25050a68e52a None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:36.972024 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc18268-f40a-40d2-bebf-25050a68e52a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:36.972254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc18268-f40a-40d2-bebf-25050a68e52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:36.997465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-bdc18268-f40a-40d2-bebf-25050a68e52a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:36.998816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-bdc18268-f40a-40d2-bebf-25050a68e52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:37.033461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2852aeb4-1786-45b8-ae58-e458f2f7634a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:37.036293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:37.073061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412af69a-4f78-439e-ad69-f51eb94cc502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:37.074322 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412af69a-4f78-439e-ad69-f51eb94cc502 None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:37.074780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412af69a-4f78-439e-ad69-f51eb94cc502 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:37.075271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412af69a-4f78-439e-ad69-f51eb94cc502 None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe/users/617786f5e72e4f638898771fb6ca88fc/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:37.075854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:37.078034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412af69a-4f78-439e-ad69-f51eb94cc502 None admin] RBAC: Authorizing `identity:create_grant(project_id=082a3596b7724c4f8f59b2687c2ca0fe, user_id=617786f5e72e4f638898771fb6ca88fc, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:37.081083 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412af69a-4f78-439e-ad69-f51eb94cc502 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:37.085501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 124/252] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:37.086661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc18268-f40a-40d2-bebf-25050a68e52a None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:37.186005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 129/253] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:23:36 2022] PUT /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe/users/617786f5e72e4f638898771fb6ca88fc/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:23:37.194339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2cc38-f2d8-4558-8a44-eb5e2176ec67 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:37.195069 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2cc38-f2d8-4558-8a44-eb5e2176ec67 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:37.195570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2cc38-f2d8-4558-8a44-eb5e2176ec67 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:37.197573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 130/254] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:37.202864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dcd57-4269-4544-b73e-30c154cdf56b None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:37.203443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dcd57-4269-4544-b73e-30c154cdf56b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:37.203977 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942dcd57-4269-4544-b73e-30c154cdf56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:37.205254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc18268-f40a-40d2-bebf-25050a68e52a None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:37.237136 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-942dcd57-4269-4544-b73e-30c154cdf56b None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:37.238732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-942dcd57-4269-4544-b73e-30c154cdf56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:37.274505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77398c7a-76b7-427d-b932-762583766953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:37.275633 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77398c7a-76b7-427d-b932-762583766953 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:37.277868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77398c7a-76b7-427d-b932-762583766953 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:37.277868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77398c7a-76b7-427d-b932-762583766953 None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:37.286903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77398c7a-76b7-427d-b932-762583766953 None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:37.289236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77398c7a-76b7-427d-b932-762583766953 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:37.292261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 125/255] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:37 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:37.299803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77398c7a-76b7-427d-b932-762583766953 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:37.395094 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 131/256] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:37.418692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:37.462405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77398c7a-76b7-427d-b932-762583766953 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:37.499264 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033761fe-486a-4539-bd46-c826416382b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:37.500157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033761fe-486a-4539-bd46-c826416382b8 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:37.500564 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033761fe-486a-4539-bd46-c826416382b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:37.501532 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033761fe-486a-4539-bd46-c826416382b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:37.502447 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033761fe-486a-4539-bd46-c826416382b8 None admin] RBAC: Adding query filter params (role.id=c8fdda710e4f416380f2e8d61fa5b71f, scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe, user.id=712d0461c7c546ca87ccd5e92c2ae1be) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:37.502887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033761fe-486a-4539-bd46-c826416382b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:37.504469 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033761fe-486a-4539-bd46-c826416382b8 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:37.510581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 126/257] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:37 2022] GET /identity/v3/role_assignments?user.id=712d0461c7c546ca87ccd5e92c2ae1be&scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe&role.id=c8fdda710e4f416380f2e8d61fa5b71f => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:37.514120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dcd57-4269-4544-b73e-30c154cdf56b None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:37.628566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:37.639351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942dcd57-4269-4544-b73e-30c154cdf56b None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:37.710502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498317d-bf3f-4912-930e-0d3da0d0a268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:37.711417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498317d-bf3f-4912-930e-0d3da0d0a268 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:37.711830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498317d-bf3f-4912-930e-0d3da0d0a268 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:37.712231 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498317d-bf3f-4912-930e-0d3da0d0a268 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:37.725946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498317d-bf3f-4912-930e-0d3da0d0a268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-5498317d-bf3f-4912-930e-0d3da0d0a268 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:37.731194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:37.732281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:37.733312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:37.733312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:37.733312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:37.733312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:37.733312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:37.733312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:37.733312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:37.733312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:37.733312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:37.733312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:37.733312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:37.733312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:37.736301 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 132/258] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:23:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:37.741795 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033761fe-486a-4539-bd46-c826416382b8 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:37.744344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498317d-bf3f-4912-930e-0d3da0d0a268 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:37.922330 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033761fe-486a-4539-bd46-c826416382b8 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:37.941202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498317d-bf3f-4912-930e-0d3da0d0a268 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:37.971120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:37.980822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c83ce-83a2-4950-9dce-fab738c2f809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:37.982227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c83ce-83a2-4950-9dce-fab738c2f809 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:37.989168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c83ce-83a2-4950-9dce-fab738c2f809 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:37.989168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c83ce-83a2-4950-9dce-fab738c2f809 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:37.989168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c83ce-83a2-4950-9dce-fab738c2f809 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:37.989168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c83ce-83a2-4950-9dce-fab738c2f809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:37.989168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c83ce-83a2-4950-9dce-fab738c2f809 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:37.999966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 127/259] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:38.005547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c83ce-83a2-4950-9dce-fab738c2f809 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:38.015457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817f311-3313-4c1e-9084-96c22fe4dcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:38.016094 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817f311-3313-4c1e-9084-96c22fe4dcfd None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:38.017187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817f311-3313-4c1e-9084-96c22fe4dcfd None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:38.017456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817f311-3313-4c1e-9084-96c22fe4dcfd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:38.036867 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817f311-3313-4c1e-9084-96c22fe4dcfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-8817f311-3313-4c1e-9084-96c22fe4dcfd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:38.044875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:38.046097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:38.047046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:38.047046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:38.047046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:38.047046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:38.047046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:38.047046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:38.047046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:38.047046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:38.047046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:38.047046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:38.047046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:38.047046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:38.047046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 133/260] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:23:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:23:38.054024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817f311-3313-4c1e-9084-96c22fe4dcfd None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:38.084589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:38.207907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c83ce-83a2-4950-9dce-fab738c2f809 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:38.210502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817f311-3313-4c1e-9084-96c22fe4dcfd None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:38.246252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f281a-5f5c-4de2-9d97-440063895721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:38.246864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f281a-5f5c-4de2-9d97-440063895721 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:38.247138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f281a-5f5c-4de2-9d97-440063895721 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:38.247640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f281a-5f5c-4de2-9d97-440063895721 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:38.248394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f281a-5f5c-4de2-9d97-440063895721 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:38.248821 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f281a-5f5c-4de2-9d97-440063895721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:38.250296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f281a-5f5c-4de2-9d97-440063895721 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:38.259427 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 134/261] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:23:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:38.265035 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f281a-5f5c-4de2-9d97-440063895721 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:38.286537 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1427be-e597-43f0-8e36-5f68f2a05807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:38.287193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1427be-e597-43f0-8e36-5f68f2a05807 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:38.287468 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1427be-e597-43f0-8e36-5f68f2a05807 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:38.287735 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1427be-e597-43f0-8e36-5f68f2a05807 None admin] PATH_INFO: `/v3/users/4f1bb48567b146d4a0b3c1ee3c7d8e11` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:38.288961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1427be-e597-43f0-8e36-5f68f2a05807 None admin] RBAC: Authorizing `identity:get_user(user_id=4f1bb48567b146d4a0b3c1ee3c7d8e11)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:38.290499 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1427be-e597-43f0-8e36-5f68f2a05807 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:38.292249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 128/262] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:38 2022] GET /identity/v3/users/4f1bb48567b146d4a0b3c1ee3c7d8e11 => generated 283 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:38.312252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1427be-e597-43f0-8e36-5f68f2a05807 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:38.348110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:38.374265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f281a-5f5c-4de2-9d97-440063895721 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:38.411617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2f1ad-a064-4470-863d-8cf919d58d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:38.412259 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2f1ad-a064-4470-863d-8cf919d58d3d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:38.412528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2f1ad-a064-4470-863d-8cf919d58d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:38.412792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e2f1ad-a064-4470-863d-8cf919d58d3d None admin] PATH_INFO: `/v3/users/4f1bb48567b146d4a0b3c1ee3c7d8e11` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:38.413976 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2f1ad-a064-4470-863d-8cf919d58d3d None admin] RBAC: Authorizing `identity:get_user(user_id=4f1bb48567b146d4a0b3c1ee3c7d8e11)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:38.415479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e2f1ad-a064-4470-863d-8cf919d58d3d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:38.422058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 135/263] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:38 2022] GET /identity/v3/users/4f1bb48567b146d4a0b3c1ee3c7d8e11 => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:38.423944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2f1ad-a064-4470-863d-8cf919d58d3d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:38.435689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:38.518833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1427be-e597-43f0-8e36-5f68f2a05807 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:38.543725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e2f1ad-a064-4470-863d-8cf919d58d3d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:38.584862 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bc43d-39ef-4e3a-b794-2633a9ac71c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:38.585467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bc43d-39ef-4e3a-b794-2633a9ac71c1 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:38.585738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bc43d-39ef-4e3a-b794-2633a9ac71c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:38.586033 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bc43d-39ef-4e3a-b794-2633a9ac71c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:38.586485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58357c1e-b1e2-4be1-92c5-4e360a3e9404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:38.587433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58357c1e-b1e2-4be1-92c5-4e360a3e9404 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:38.587893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58357c1e-b1e2-4be1-92c5-4e360a3e9404 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:38.593898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58357c1e-b1e2-4be1-92c5-4e360a3e9404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:38.601680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:38.673903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bc43d-39ef-4e3a-b794-2633a9ac71c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:38.675639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bc43d-39ef-4e3a-b794-2633a9ac71c1 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:38.676309 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58357c1e-b1e2-4be1-92c5-4e360a3e9404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:38.678355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58357c1e-b1e2-4be1-92c5-4e360a3e9404 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:38.746602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:38.800753 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 136/264] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 05:23:38.806459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bc43d-39ef-4e3a-b794-2633a9ac71c1 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:38.834963 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 129/265] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:38.857545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:38.859495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58357c1e-b1e2-4be1-92c5-4e360a3e9404 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:38.926317 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bc43d-39ef-4e3a-b794-2633a9ac71c1 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:38.979380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7801a-956d-4f90-8c6c-4fe65ccf8443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:38.980011 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7801a-956d-4f90-8c6c-4fe65ccf8443 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:38.980281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7801a-956d-4f90-8c6c-4fe65ccf8443 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:38.980545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c7801a-956d-4f90-8c6c-4fe65ccf8443 None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:38.983630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7801a-956d-4f90-8c6c-4fe65ccf8443 None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:38.987438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c7801a-956d-4f90-8c6c-4fe65ccf8443 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:38.989081 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 137/266] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:38 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:39.003872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7801a-956d-4f90-8c6c-4fe65ccf8443 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:39.078482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58357c1e-b1e2-4be1-92c5-4e360a3e9404 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:39.149493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5938a-c42e-4d6e-be22-088e886113da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:39.150152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5938a-c42e-4d6e-be22-088e886113da None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:39.150421 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5938a-c42e-4d6e-be22-088e886113da None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:39.150684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5938a-c42e-4d6e-be22-088e886113da None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:39.151796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5938a-c42e-4d6e-be22-088e886113da None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:39.153222 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5938a-c42e-4d6e-be22-088e886113da None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:39.154866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 130/267] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:38 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:39.178090 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5938a-c42e-4d6e-be22-088e886113da None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:39.209776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c7801a-956d-4f90-8c6c-4fe65ccf8443 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:39.242175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:39.258547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:39.313065 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5938a-c42e-4d6e-be22-088e886113da None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:39.347918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861df63-3692-49e8-a2b2-2bf258fa1fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:39.348560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861df63-3692-49e8-a2b2-2bf258fa1fab None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:39.348940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861df63-3692-49e8-a2b2-2bf258fa1fab None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:39.349213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861df63-3692-49e8-a2b2-2bf258fa1fab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:39.349848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14353ed7-5536-481e-8dff-f60793987abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:39.349966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861df63-3692-49e8-a2b2-2bf258fa1fab None admin] RBAC: Adding query filter params (role.id=93f699f03f0f41498415ab9533bdfcb4, scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe, user.id=4f1bb48567b146d4a0b3c1ee3c7d8e11) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:39.350265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861df63-3692-49e8-a2b2-2bf258fa1fab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:39.350461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14353ed7-5536-481e-8dff-f60793987abf None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:39.350734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14353ed7-5536-481e-8dff-f60793987abf None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:39.350995 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14353ed7-5536-481e-8dff-f60793987abf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:39.351709 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14353ed7-5536-481e-8dff-f60793987abf None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe, user.id=4f1bb48567b146d4a0b3c1ee3c7d8e11) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:39.351808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861df63-3692-49e8-a2b2-2bf258fa1fab None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:39.352005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14353ed7-5536-481e-8dff-f60793987abf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:39.353417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14353ed7-5536-481e-8dff-f60793987abf None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:39.359626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 131/268] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:39 2022] GET /identity/v3/role_assignments?user.id=4f1bb48567b146d4a0b3c1ee3c7d8e11&scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:39.376645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 138/269] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:39 2022] GET /identity/v3/role_assignments?user.id=4f1bb48567b146d4a0b3c1ee3c7d8e11&scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe&role.id=93f699f03f0f41498415ab9533bdfcb4 => generated 608 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:39.884341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e5f2-7fbc-42b6-bf25-1948ae42b0ca None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:39.884700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e5f2-7fbc-42b6-bf25-1948ae42b0ca None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:39.884864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e5f2-7fbc-42b6-bf25-1948ae42b0ca None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:39.886270 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 132/270] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:39.892034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcefefc-dbe0-4042-9359-bbf2d4f87dee None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:39.892508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcefefc-dbe0-4042-9359-bbf2d4f87dee None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:39.892784 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcefefc-dbe0-4042-9359-bbf2d4f87dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:39.929576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-ffcefefc-dbe0-4042-9359-bbf2d4f87dee None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:39.930822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-ffcefefc-dbe0-4042-9359-bbf2d4f87dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:40.021877 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 139/271] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:40.214526 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb1823-b394-4ce3-9ce6-ed5764a665ce None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:40.214821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb1823-b394-4ce3-9ce6-ed5764a665ce None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:40.215001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb1823-b394-4ce3-9ce6-ed5764a665ce None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:40.216576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 133/272] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:40.222172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d2adf-2b3c-499f-809e-ee36aeea904a None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:40.222455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d2adf-2b3c-499f-809e-ee36aeea904a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:40.222728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d2adf-2b3c-499f-809e-ee36aeea904a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:40.242137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0d8d2adf-2b3c-499f-809e-ee36aeea904a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:40.243366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0d8d2adf-2b3c-499f-809e-ee36aeea904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:40.286027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f024dd5-f6d4-45db-a6dd-5c07ffd64079 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:40.286985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f024dd5-f6d4-45db-a6dd-5c07ffd64079 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:40.287452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f024dd5-f6d4-45db-a6dd-5c07ffd64079 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:40.289221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 134/273] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:23:40.294400 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d3b62-40ac-4752-864e-47a1d784722d None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:40.294922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d3b62-40ac-4752-864e-47a1d784722d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:40.295388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d3b62-40ac-4752-864e-47a1d784722d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:40.314400 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-b85d3b62-40ac-4752-864e-47a1d784722d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:40.315928 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b85d3b62-40ac-4752-864e-47a1d784722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:40.317134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 140/274] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:40.317998 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d2adf-2b3c-499f-809e-ee36aeea904a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:40.361800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:40.405890 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 135/275] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:40.440152 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d2adf-2b3c-499f-809e-ee36aeea904a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:40.479051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705190ae-a108-4f51-b696-37d1388b177c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:40.480046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705190ae-a108-4f51-b696-37d1388b177c None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:40.480544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705190ae-a108-4f51-b696-37d1388b177c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:40.481030 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705190ae-a108-4f51-b696-37d1388b177c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:40.494925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705190ae-a108-4f51-b696-37d1388b177c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-705190ae-a108-4f51-b696-37d1388b177c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:40.500469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:40.502746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:40.503620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:40.503620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:40.503620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:40.503620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:40.503620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:40.503620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:40.503620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:40.503620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:40.503620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:40.503620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:40.503620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:40.503620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:40.506344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 141/276] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:40.514164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d3b62-40ac-4752-864e-47a1d784722d None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:40.535121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:40.590897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae3391-5516-4e9b-b906-7895990e920c None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:40.591180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae3391-5516-4e9b-b906-7895990e920c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:40.591458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae3391-5516-4e9b-b906-7895990e920c None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:40.592995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 142/277] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:23:40.598907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329ca87-0f49-4cba-91db-0ed282e903f9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:40.599189 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329ca87-0f49-4cba-91db-0ed282e903f9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:40.599463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329ca87-0f49-4cba-91db-0ed282e903f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:40.617205 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d3b62-40ac-4752-864e-47a1d784722d None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:40.618231 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-8329ca87-0f49-4cba-91db-0ed282e903f9 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:40.619421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8329ca87-0f49-4cba-91db-0ed282e903f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:40.653514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21505256-0cc1-4204-be0b-bd4b15fa99a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:40.654369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21505256-0cc1-4204-be0b-bd4b15fa99a2 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:40.654786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21505256-0cc1-4204-be0b-bd4b15fa99a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:40.655188 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21505256-0cc1-4204-be0b-bd4b15fa99a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:40.656018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21505256-0cc1-4204-be0b-bd4b15fa99a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:40.656465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21505256-0cc1-4204-be0b-bd4b15fa99a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:40.658075 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21505256-0cc1-4204-be0b-bd4b15fa99a2 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:40.670796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 136/278] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:40.671656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21505256-0cc1-4204-be0b-bd4b15fa99a2 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:40.690002 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 143/279] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:40.694549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329ca87-0f49-4cba-91db-0ed282e903f9 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:40.723725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:40.758926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:40.779212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21505256-0cc1-4204-be0b-bd4b15fa99a2 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:40.804238 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329ca87-0f49-4cba-91db-0ed282e903f9 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:40.817582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6f5ac-dfd4-4a60-a475-abafa290333f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:40.818393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6f5ac-dfd4-4a60-a475-abafa290333f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:40.818676 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6f5ac-dfd4-4a60-a475-abafa290333f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:40.818946 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6f5ac-dfd4-4a60-a475-abafa290333f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:40.820413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6f5ac-dfd4-4a60-a475-abafa290333f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:40.821871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e6f5ac-dfd4-4a60-a475-abafa290333f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:40.823592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 137/280] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:40.824187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6f5ac-dfd4-4a60-a475-abafa290333f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:40.845107 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69522c73-71f4-4e6b-9739-90f6c72ace91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:40.845732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69522c73-71f4-4e6b-9739-90f6c72ace91 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:40.846025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69522c73-71f4-4e6b-9739-90f6c72ace91 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:40.846284 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69522c73-71f4-4e6b-9739-90f6c72ace91 None admin] PATH_INFO: `/v3/users/9673ba8598f848c2a6f78d4ac5da7bcf` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:40.860093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69522c73-71f4-4e6b-9739-90f6c72ace91 None admin] RBAC: Authorizing `identity:get_user(user_id=9673ba8598f848c2a6f78d4ac5da7bcf)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:40.861704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69522c73-71f4-4e6b-9739-90f6c72ace91 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:40.863390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 144/281] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:40 2022] GET /identity/v3/users/9673ba8598f848c2a6f78d4ac5da7bcf => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:40.864121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69522c73-71f4-4e6b-9739-90f6c72ace91 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:40.945778 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e6f5ac-dfd4-4a60-a475-abafa290333f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:40.957933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:40.975879 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69522c73-71f4-4e6b-9739-90f6c72ace91 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:40.980198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b81e0-c16a-49cd-9018-44cce492da89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:40.980822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b81e0-c16a-49cd-9018-44cce492da89 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:40.981096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b81e0-c16a-49cd-9018-44cce492da89 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:40.981366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b81e0-c16a-49cd-9018-44cce492da89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:40.992690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b81e0-c16a-49cd-9018-44cce492da89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-890b81e0-c16a-49cd-9018-44cce492da89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:40.997997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:40.999073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 138/282] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:40.999961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b81e0-c16a-49cd-9018-44cce492da89 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:41.013167 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:41.013773 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac775bec-8692-434c-9704-95716677c73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:41.014404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac775bec-8692-434c-9704-95716677c73a None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:41.014675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac775bec-8692-434c-9704-95716677c73a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:41.014938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac775bec-8692-434c-9704-95716677c73a None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:41.015586 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac775bec-8692-434c-9704-95716677c73a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:41.017524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac775bec-8692-434c-9704-95716677c73a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:41.097450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b81e0-c16a-49cd-9018-44cce492da89 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:41.104803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 145/283] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:23:40 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:41.105612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac775bec-8692-434c-9704-95716677c73a None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:41.133151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3131d9d-d0ca-4548-a885-fb3e6cb11ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:41.133761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3131d9d-d0ca-4548-a885-fb3e6cb11ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:41.134068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3131d9d-d0ca-4548-a885-fb3e6cb11ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:41.134332 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3131d9d-d0ca-4548-a885-fb3e6cb11ee1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:41.135079 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3131d9d-d0ca-4548-a885-fb3e6cb11ee1 None admin] RBAC: Adding query filter params (role.id=413cc5f8baab440bab827a3547b65f33, user.id=9673ba8598f848c2a6f78d4ac5da7bcf, scope.system=all) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:41.135380 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3131d9d-d0ca-4548-a885-fb3e6cb11ee1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:41.136842 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3131d9d-d0ca-4548-a885-fb3e6cb11ee1 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:41.142838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 139/284] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Sep 16 05:23:40 2022] GET /identity/v3/role_assignments?user.id=9673ba8598f848c2a6f78d4ac5da7bcf&scope.system=all&role.id=413cc5f8baab440bab827a3547b65f33 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:41.143696 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3131d9d-d0ca-4548-a885-fb3e6cb11ee1 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:41.249623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:41.272909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac775bec-8692-434c-9704-95716677c73a None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:41.282598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3131d9d-d0ca-4548-a885-fb3e6cb11ee1 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:41.323143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c7955-0342-4d7d-b4c4-0d606076c97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:41.323794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c7955-0342-4d7d-b4c4-0d606076c97b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:41.324067 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c7955-0342-4d7d-b4c4-0d606076c97b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:41.324329 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c7955-0342-4d7d-b4c4-0d606076c97b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:41.327414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03192320-f939-4118-8816-dcc54ce1bd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:41.327414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192320-f939-4118-8816-dcc54ce1bd12 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:41.328002 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192320-f939-4118-8816-dcc54ce1bd12 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:41.328002 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192320-f939-4118-8816-dcc54ce1bd12 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:41.330469 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03192320-f939-4118-8816-dcc54ce1bd12 None admin] RBAC: Adding query filter params (name=member) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:41.330469 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03192320-f939-4118-8816-dcc54ce1bd12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:41.331764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03192320-f939-4118-8816-dcc54ce1bd12 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:41.339981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:41.341408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 140/285] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:41.347823 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03192320-f939-4118-8816-dcc54ce1bd12 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:41.349296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c7955-0342-4d7d-b4c4-0d606076c97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-f12c7955-0342-4d7d-b4c4-0d606076c97b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:41.354614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:41.355476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:41.356487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:41.356487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:41.356487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:41.356487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:41.356487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:41.356487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:41.356487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:41.356487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:41.356487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:41.356487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:41.356487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:41.356487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:41.356487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 146/286] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:41.362393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c7955-0342-4d7d-b4c4-0d606076c97b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:41.469288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03192320-f939-4118-8816-dcc54ce1bd12 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:41.485582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c7955-0342-4d7d-b4c4-0d606076c97b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:41.507050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22aaf2c-1ea6-417a-ad3e-82fa8744597d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:41.507860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22aaf2c-1ea6-417a-ad3e-82fa8744597d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:41.507970 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22aaf2c-1ea6-417a-ad3e-82fa8744597d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:41.508214 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22aaf2c-1ea6-417a-ad3e-82fa8744597d None admin] PATH_INFO: `/v3/users/617786f5e72e4f638898771fb6ca88fc` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:41.509370 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22aaf2c-1ea6-417a-ad3e-82fa8744597d None admin] RBAC: Authorizing `identity:get_user(user_id=617786f5e72e4f638898771fb6ca88fc)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:41.510951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22aaf2c-1ea6-417a-ad3e-82fa8744597d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:41.512446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 141/287] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:41 2022] GET /identity/v3/users/617786f5e72e4f638898771fb6ca88fc => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:41.518236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22aaf2c-1ea6-417a-ad3e-82fa8744597d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:41.522082 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:41.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd1372-e0e0-4c32-a60b-f81ac9bccf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:41.524732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd1372-e0e0-4c32-a60b-f81ac9bccf10 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:41.525144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd1372-e0e0-4c32-a60b-f81ac9bccf10 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:41.525558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd1372-e0e0-4c32-a60b-f81ac9bccf10 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:41.526398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd1372-e0e0-4c32-a60b-f81ac9bccf10 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:41.526831 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd1372-e0e0-4c32-a60b-f81ac9bccf10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:41.528437 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd1372-e0e0-4c32-a60b-f81ac9bccf10 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:41.538135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 147/288] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:41.543743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd1372-e0e0-4c32-a60b-f81ac9bccf10 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:41.620081 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:41.650062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22aaf2c-1ea6-417a-ad3e-82fa8744597d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:41.670417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd1372-e0e0-4c32-a60b-f81ac9bccf10 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:41.692041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d33fe-b0bb-4682-a226-107fb1412d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:41.692803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d33fe-b0bb-4682-a226-107fb1412d63 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:41.693174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d33fe-b0bb-4682-a226-107fb1412d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:41.693536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d33fe-b0bb-4682-a226-107fb1412d63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:41.714549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90058503-2568-4f16-9451-b4a4e80682f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:41.715173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90058503-2568-4f16-9451-b4a4e80682f4 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:41.715442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90058503-2568-4f16-9451-b4a4e80682f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:41.715705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90058503-2568-4f16-9451-b4a4e80682f4 None admin] PATH_INFO: `/v3/users/712d0461c7c546ca87ccd5e92c2ae1be` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:41.717433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90058503-2568-4f16-9451-b4a4e80682f4 None admin] RBAC: Authorizing `identity:get_user(user_id=712d0461c7c546ca87ccd5e92c2ae1be)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:41.719336 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90058503-2568-4f16-9451-b4a4e80682f4 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:41.722193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 148/289] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:41 2022] GET /identity/v3/users/712d0461c7c546ca87ccd5e92c2ae1be => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:41.730747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90058503-2568-4f16-9451-b4a4e80682f4 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:41.796255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d33fe-b0bb-4682-a226-107fb1412d63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:41.798779 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d33fe-b0bb-4682-a226-107fb1412d63 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:41.808965 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:41.917237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90058503-2568-4f16-9451-b4a4e80682f4 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:41.980597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 142/290] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 05:23:41.986142 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d33fe-b0bb-4682-a226-107fb1412d63 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:41.991175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:42.017697 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f32e4-a722-47ae-9854-89077d78114d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:42.018368 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f32e4-a722-47ae-9854-89077d78114d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:42.018637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f32e4-a722-47ae-9854-89077d78114d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:42.018906 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f32e4-a722-47ae-9854-89077d78114d None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:42.021045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f32e4-a722-47ae-9854-89077d78114d None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:42.023645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72f32e4-a722-47ae-9854-89077d78114d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:42.026327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 149/291] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:41 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:42.034692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f32e4-a722-47ae-9854-89077d78114d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:42.104984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d33fe-b0bb-4682-a226-107fb1412d63 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:42.107626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:42.141926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a81e06-ad3a-4c8d-ba42-bfabc9ca3fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:42.142828 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a81e06-ad3a-4c8d-ba42-bfabc9ca3fdd None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:42.143245 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a81e06-ad3a-4c8d-ba42-bfabc9ca3fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:42.143724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a81e06-ad3a-4c8d-ba42-bfabc9ca3fdd None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:42.145762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a81e06-ad3a-4c8d-ba42-bfabc9ca3fdd None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:42.147479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a81e06-ad3a-4c8d-ba42-bfabc9ca3fdd None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:42.149622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 143/292] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:41 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:42.161596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a81e06-ad3a-4c8d-ba42-bfabc9ca3fdd None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:42.271495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a81e06-ad3a-4c8d-ba42-bfabc9ca3fdd None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:42.301623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72f32e4-a722-47ae-9854-89077d78114d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:42.314760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb30992-8db5-44c6-af63-f1707cab43fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:42.315370 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb30992-8db5-44c6-af63-f1707cab43fe None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:42.315645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb30992-8db5-44c6-af63-f1707cab43fe None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:42.315911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb30992-8db5-44c6-af63-f1707cab43fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:42.316639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb30992-8db5-44c6-af63-f1707cab43fe None admin] RBAC: Adding query filter params (role.id=413cc5f8baab440bab827a3547b65f33, scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe, user.id=617786f5e72e4f638898771fb6ca88fc) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:42.316941 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb30992-8db5-44c6-af63-f1707cab43fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:42.318422 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb30992-8db5-44c6-af63-f1707cab43fe None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:42.331108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 144/293] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:42 2022] GET /identity/v3/role_assignments?user.id=617786f5e72e4f638898771fb6ca88fc&scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe&role.id=413cc5f8baab440bab827a3547b65f33 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:42.370802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c994c-2705-4266-bc82-aacd3b5d8031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:42.371503 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c994c-2705-4266-bc82-aacd3b5d8031 None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:42.371779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c994c-2705-4266-bc82-aacd3b5d8031 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:42.372051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c994c-2705-4266-bc82-aacd3b5d8031 None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe/users/712d0461c7c546ca87ccd5e92c2ae1be/roles/c8fdda710e4f416380f2e8d61fa5b71f` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:42.374281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c994c-2705-4266-bc82-aacd3b5d8031 None admin] RBAC: Authorizing `identity:create_grant(project_id=082a3596b7724c4f8f59b2687c2ca0fe, user_id=712d0461c7c546ca87ccd5e92c2ae1be, role_id=c8fdda710e4f416380f2e8d61fa5b71f)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:42.376332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c994c-2705-4266-bc82-aacd3b5d8031 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:42.468228 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 150/294] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:23:42 2022] PUT /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe/users/712d0461c7c546ca87ccd5e92c2ae1be/roles/c8fdda710e4f416380f2e8d61fa5b71f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:23:43.540033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbea7a-65a3-4a2f-934b-9a293d4b8fb0 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:43.540551 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbea7a-65a3-4a2f-934b-9a293d4b8fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:43.540551 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbea7a-65a3-4a2f-934b-9a293d4b8fb0 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:43.542141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 145/295] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:43.547443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a483076-e11b-4007-9b5a-be3581a298a5 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:43.547836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a483076-e11b-4007-9b5a-be3581a298a5 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:43.548015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a483076-e11b-4007-9b5a-be3581a298a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:43.566512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-5a483076-e11b-4007-9b5a-be3581a298a5 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:43.567693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5a483076-e11b-4007-9b5a-be3581a298a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:43.618453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:43.635372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 151/296] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:43.811505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d214b-b65a-4755-9f50-56b73cc7f94e None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:43.811505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d214b-b65a-4755-9f50-56b73cc7f94e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:43.811505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d214b-b65a-4755-9f50-56b73cc7f94e None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:43.811505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 146/297] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:43.818539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895465e-1cc4-4b58-8ccb-182cda491fa1 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:43.819060 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895465e-1cc4-4b58-8ccb-182cda491fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:43.819520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895465e-1cc4-4b58-8ccb-182cda491fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:43.838245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-4895465e-1cc4-4b58-8ccb-182cda491fa1 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:43.839746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4895465e-1cc4-4b58-8ccb-182cda491fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:43.840952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87416e62-afb3-4389-aaec-f2c86f12e926 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:43.841465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87416e62-afb3-4389-aaec-f2c86f12e926 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:43.841973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87416e62-afb3-4389-aaec-f2c86f12e926 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:43.843733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 147/298] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:43.849034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56a3fc-85fa-448e-985a-eea39156feb5 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:43.849591 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56a3fc-85fa-448e-985a-eea39156feb5 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:43.850088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56a3fc-85fa-448e-985a-eea39156feb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:43.869398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-cc56a3fc-85fa-448e-985a-eea39156feb5 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:43.871094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-cc56a3fc-85fa-448e-985a-eea39156feb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:43.903423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:43.909408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 152/299] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:43.910290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895465e-1cc4-4b58-8ccb-182cda491fa1 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:43.947518 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 148/300] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:44.021538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895465e-1cc4-4b58-8ccb-182cda491fa1 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:44.038308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:44.058653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de5e95-ff19-4dec-a581-74790a9d3191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:44.059270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de5e95-ff19-4dec-a581-74790a9d3191 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:44.059591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de5e95-ff19-4dec-a581-74790a9d3191 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:44.059860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de5e95-ff19-4dec-a581-74790a9d3191 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:44.070648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de5e95-ff19-4dec-a581-74790a9d3191 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-54de5e95-ff19-4dec-a581-74790a9d3191 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:44.075621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:44.076498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:44.077297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:44.077297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:44.077297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:44.077297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:44.077297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:44.077297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:44.077297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:44.077297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:44.077297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:44.077297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:44.077297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:44.077297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:44.083039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 153/301] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:44.089006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56a3fc-85fa-448e-985a-eea39156feb5 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:44.192595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56a3fc-85fa-448e-985a-eea39156feb5 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:44.206397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de5e95-ff19-4dec-a581-74790a9d3191 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:44.229723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0370a2-98de-4b21-86ed-5320bd1cdef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:44.230363 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0370a2-98de-4b21-86ed-5320bd1cdef1 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:44.230639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0370a2-98de-4b21-86ed-5320bd1cdef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:44.230908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0370a2-98de-4b21-86ed-5320bd1cdef1 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:44.231776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0370a2-98de-4b21-86ed-5320bd1cdef1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:44.232082 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0370a2-98de-4b21-86ed-5320bd1cdef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:44.233521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0370a2-98de-4b21-86ed-5320bd1cdef1 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:44.247563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 149/302] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:44.249364 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0370a2-98de-4b21-86ed-5320bd1cdef1 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:44.283304 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:44.330181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de5e95-ff19-4dec-a581-74790a9d3191 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:44.366083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0370a2-98de-4b21-86ed-5320bd1cdef1 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:44.367520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c4992-a0e3-48f4-9c70-1eb9453aea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:44.368146 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c4992-a0e3-48f4-9c70-1eb9453aea6f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:44.368417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c4992-a0e3-48f4-9c70-1eb9453aea6f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:44.368682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c4992-a0e3-48f4-9c70-1eb9453aea6f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:44.379706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1c4992-a0e3-48f4-9c70-1eb9453aea6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-5d1c4992-a0e3-48f4-9c70-1eb9453aea6f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:44.384676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:44.385746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:44.386852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:44.386852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:44.386852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:44.386852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:44.386852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:44.386852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:44.386852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:44.386852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:44.386852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:44.386852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:44.386852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:44.386852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:44.390878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 154/303] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:23:44.392520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c4992-a0e3-48f4-9c70-1eb9453aea6f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:44.412279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e1ebc-08e1-4761-bd8a-44ee4ac43635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:44.412906 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e1ebc-08e1-4761-bd8a-44ee4ac43635 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:44.413210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e1ebc-08e1-4761-bd8a-44ee4ac43635 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:44.413445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e1ebc-08e1-4761-bd8a-44ee4ac43635 None admin] PATH_INFO: `/v3/users/9673ba8598f848c2a6f78d4ac5da7bcf` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:44.414622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e1ebc-08e1-4761-bd8a-44ee4ac43635 None admin] RBAC: Authorizing `identity:get_user(user_id=9673ba8598f848c2a6f78d4ac5da7bcf)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:44.416041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e1ebc-08e1-4761-bd8a-44ee4ac43635 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:44.417889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 150/304] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:44 2022] GET /identity/v3/users/9673ba8598f848c2a6f78d4ac5da7bcf => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:44.418535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e1ebc-08e1-4761-bd8a-44ee4ac43635 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:44.434162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:44.504135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c4992-a0e3-48f4-9c70-1eb9453aea6f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:44.515156 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:44.521445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e1ebc-08e1-4761-bd8a-44ee4ac43635 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:44.540118 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89b665-1667-4f1a-9b62-646b5aa23125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:44.540739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89b665-1667-4f1a-9b62-646b5aa23125 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:44.541017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89b665-1667-4f1a-9b62-646b5aa23125 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:44.541276 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89b665-1667-4f1a-9b62-646b5aa23125 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:44.542395 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89b665-1667-4f1a-9b62-646b5aa23125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:44.543946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89b665-1667-4f1a-9b62-646b5aa23125 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:44.545495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 155/305] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:44.546403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89b665-1667-4f1a-9b62-646b5aa23125 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:44.565543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575a7e9-e860-4eed-afd3-4d894d3a55eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:44.566194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575a7e9-e860-4eed-afd3-4d894d3a55eb None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:44.566473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575a7e9-e860-4eed-afd3-4d894d3a55eb None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:44.566742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575a7e9-e860-4eed-afd3-4d894d3a55eb None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:44.567407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575a7e9-e860-4eed-afd3-4d894d3a55eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:44.567706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575a7e9-e860-4eed-afd3-4d894d3a55eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:44.569069 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575a7e9-e860-4eed-afd3-4d894d3a55eb None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:44.577297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 151/306] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:44.578117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575a7e9-e860-4eed-afd3-4d894d3a55eb None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:44.665725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89b665-1667-4f1a-9b62-646b5aa23125 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:44.669385 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:44.698722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575a7e9-e860-4eed-afd3-4d894d3a55eb None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:44.703929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ec330-ac86-4d84-9862-7260a4bd1d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:44.704583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ec330-ac86-4d84-9862-7260a4bd1d3c None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:44.704856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ec330-ac86-4d84-9862-7260a4bd1d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:44.705130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ec330-ac86-4d84-9862-7260a4bd1d3c None admin] PATH_INFO: `/v3/system/users/9673ba8598f848c2a6f78d4ac5da7bcf/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:44.706631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ec330-ac86-4d84-9862-7260a4bd1d3c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9673ba8598f848c2a6f78d4ac5da7bcf, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:44.707996 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ec330-ac86-4d84-9862-7260a4bd1d3c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:44.732955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad78db-2821-4bae-952d-819a49cedead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:44.733481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad78db-2821-4bae-952d-819a49cedead None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:44.733754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad78db-2821-4bae-952d-819a49cedead None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:44.734054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad78db-2821-4bae-952d-819a49cedead None admin] PATH_INFO: `/v3/groups` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:44.734678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad78db-2821-4bae-952d-819a49cedead None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:44.737178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad78db-2821-4bae-952d-819a49cedead None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:44.788262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 156/307] 10.222.0.49 () {58 vars in 1334 bytes} [Fri Sep 16 05:23:44 2022] PUT /identity/v3/system/users/9673ba8598f848c2a6f78d4ac5da7bcf/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:23:44.789255 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ec330-ac86-4d84-9862-7260a4bd1d3c None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:44.822805 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 152/308] 10.222.0.49 () {60 vars in 1134 bytes} [Fri Sep 16 05:23:44 2022] POST /identity/v3/groups => generated 229 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:44.823977 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c89ecb-d1f9-48de-886d-2da4b9d36c06 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:44.824287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c89ecb-d1f9-48de-886d-2da4b9d36c06 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:44.824557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c89ecb-d1f9-48de-886d-2da4b9d36c06 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:44.831414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 153/309] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:44.832950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed08f20-9842-4fcd-9e6d-3ae847b10076 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:44.833233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed08f20-9842-4fcd-9e6d-3ae847b10076 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:44.833504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed08f20-9842-4fcd-9e6d-3ae847b10076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:44.864924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-7ed08f20-9842-4fcd-9e6d-3ae847b10076 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:44.867543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7ed08f20-9842-4fcd-9e6d-3ae847b10076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:44.889502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:44.946075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ec330-ac86-4d84-9862-7260a4bd1d3c None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:44.962263 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 154/310] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:44.998827 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a393b70-0534-4267-b16b-9e4e8d8faabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:44.999914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a393b70-0534-4267-b16b-9e4e8d8faabc None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:45.000402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a393b70-0534-4267-b16b-9e4e8d8faabc None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:45.001693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a393b70-0534-4267-b16b-9e4e8d8faabc None admin] PATH_INFO: `/v3/users/03da65d67cf94c37a9dc8c1589433ca7` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:45.018887 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a393b70-0534-4267-b16b-9e4e8d8faabc None admin] RBAC: Authorizing `identity:get_user(user_id=03da65d67cf94c37a9dc8c1589433ca7)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:45.020697 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a393b70-0534-4267-b16b-9e4e8d8faabc None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:45.022676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 157/311] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:44 2022] GET /identity/v3/users/03da65d67cf94c37a9dc8c1589433ca7 => generated 293 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:45.028481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed08f20-9842-4fcd-9e6d-3ae847b10076 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:45.138454 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed08f20-9842-4fcd-9e6d-3ae847b10076 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:45.180063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150587b3-2354-423d-bdde-6dc9e64dcc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:45.181052 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150587b3-2354-423d-bdde-6dc9e64dcc11 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:45.181524 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150587b3-2354-423d-bdde-6dc9e64dcc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:45.182015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150587b3-2354-423d-bdde-6dc9e64dcc11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:45.183114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150587b3-2354-423d-bdde-6dc9e64dcc11 None admin] RBAC: Adding query filter params (role.id=c8fdda710e4f416380f2e8d61fa5b71f, user.id=03da65d67cf94c37a9dc8c1589433ca7, scope.system=all) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:45.183615 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150587b3-2354-423d-bdde-6dc9e64dcc11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:45.185293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150587b3-2354-423d-bdde-6dc9e64dcc11 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:45.192074 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 155/312] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Sep 16 05:23:45 2022] GET /identity/v3/role_assignments?user.id=03da65d67cf94c37a9dc8c1589433ca7&scope.system=all&role.id=c8fdda710e4f416380f2e8d61fa5b71f => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:45.258517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a393b70-0534-4267-b16b-9e4e8d8faabc None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:45.289067 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:45.373705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a393b70-0534-4267-b16b-9e4e8d8faabc None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:45.409143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d32d15-52d3-4934-a04f-5814aab6957e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:45.409775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32d15-52d3-4934-a04f-5814aab6957e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:45.410075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32d15-52d3-4934-a04f-5814aab6957e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:45.410342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d32d15-52d3-4934-a04f-5814aab6957e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:45.422946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d32d15-52d3-4934-a04f-5814aab6957e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-a7d32d15-52d3-4934-a04f-5814aab6957e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:45.427845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:45.428726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:45.429527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:45.429527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:45.429527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:45.429527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:45.429527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:45.429527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:45.429527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:45.429527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:45.429527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:45.429527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:45.429527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:45.429527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:45.429527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 158/313] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:45.439885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150587b3-2354-423d-bdde-6dc9e64dcc11 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:45.454964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:45.543632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150587b3-2354-423d-bdde-6dc9e64dcc11 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:45.580833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058fbb4-8668-4642-b9b5-ea7389a029cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:45.581457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058fbb4-8668-4642-b9b5-ea7389a029cc None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:45.581707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058fbb4-8668-4642-b9b5-ea7389a029cc None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:45.582005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058fbb4-8668-4642-b9b5-ea7389a029cc None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:45.582683 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058fbb4-8668-4642-b9b5-ea7389a029cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:45.582980 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058fbb4-8668-4642-b9b5-ea7389a029cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:45.584345 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058fbb4-8668-4642-b9b5-ea7389a029cc None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:45.593379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 156/314] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:45.599227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d32d15-52d3-4934-a04f-5814aab6957e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:45.745026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:45.763022 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d32d15-52d3-4934-a04f-5814aab6957e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:45.799730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977f3e5-e259-4407-b6bb-97f289e8ba4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:45.800524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977f3e5-e259-4407-b6bb-97f289e8ba4c None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:45.800804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977f3e5-e259-4407-b6bb-97f289e8ba4c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:45.801075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977f3e5-e259-4407-b6bb-97f289e8ba4c None admin] PATH_INFO: `/v3/users/712d0461c7c546ca87ccd5e92c2ae1be` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:45.802241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977f3e5-e259-4407-b6bb-97f289e8ba4c None admin] RBAC: Authorizing `identity:get_user(user_id=712d0461c7c546ca87ccd5e92c2ae1be)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:45.803679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977f3e5-e259-4407-b6bb-97f289e8ba4c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:45.805553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 159/315] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:45 2022] GET /identity/v3/users/712d0461c7c546ca87ccd5e92c2ae1be => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:45.811065 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058fbb4-8668-4642-b9b5-ea7389a029cc None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:45.891904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:45.935936 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058fbb4-8668-4642-b9b5-ea7389a029cc None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:45.978332 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c831d-a930-4047-ad37-a5f01671f93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:45.978957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c831d-a930-4047-ad37-a5f01671f93b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:45.979234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c831d-a930-4047-ad37-a5f01671f93b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:45.979504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c831d-a930-4047-ad37-a5f01671f93b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:46.058060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c831d-a930-4047-ad37-a5f01671f93b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:46.059421 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c831d-a930-4047-ad37-a5f01671f93b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:46.136395 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:46.167762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 157/316] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:46.175178 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977f3e5-e259-4407-b6bb-97f289e8ba4c None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:46.278076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977f3e5-e259-4407-b6bb-97f289e8ba4c None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:46.319661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ea510-6964-4b16-9211-b782de58a945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:46.320202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ea510-6964-4b16-9211-b782de58a945 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:46.320467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ea510-6964-4b16-9211-b782de58a945 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:46.320727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ea510-6964-4b16-9211-b782de58a945 None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:46.321848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ea510-6964-4b16-9211-b782de58a945 None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:46.323299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54ea510-6964-4b16-9211-b782de58a945 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:46.325215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 160/317] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:46 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:46.331713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c831d-a930-4047-ad37-a5f01671f93b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:46.432496 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c831d-a930-4047-ad37-a5f01671f93b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:46.470067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29847fa8-a747-46ca-97e9-b62a9939510d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:46.470673 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29847fa8-a747-46ca-97e9-b62a9939510d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:46.470951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29847fa8-a747-46ca-97e9-b62a9939510d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:46.471217 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29847fa8-a747-46ca-97e9-b62a9939510d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:46.471938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29847fa8-a747-46ca-97e9-b62a9939510d None admin] RBAC: Adding query filter params (role.id=c8fdda710e4f416380f2e8d61fa5b71f, scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe, user.id=712d0461c7c546ca87ccd5e92c2ae1be) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:46.472240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29847fa8-a747-46ca-97e9-b62a9939510d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:46.473667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29847fa8-a747-46ca-97e9-b62a9939510d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:46.479945 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 158/318] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:23:46 2022] GET /identity/v3/role_assignments?user.id=712d0461c7c546ca87ccd5e92c2ae1be&scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe&role.id=c8fdda710e4f416380f2e8d61fa5b71f => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:47.083940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad06bb5-da24-44fd-8495-d8e2152630a9 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:47.085196 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad06bb5-da24-44fd-8495-d8e2152630a9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:47.085687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad06bb5-da24-44fd-8495-d8e2152630a9 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:47.087666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 161/319] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:47.093138 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692c505-0e41-4cee-b521-b3592ae4945f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:47.093687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692c505-0e41-4cee-b521-b3592ae4945f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:47.093687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692c505-0e41-4cee-b521-b3592ae4945f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:47.117057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c692c505-0e41-4cee-b521-b3592ae4945f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:47.118616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c692c505-0e41-4cee-b521-b3592ae4945f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:47.149153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:47.187564 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 159/320] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:47.209496 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c05e9-1361-41f7-b015-d6e0975f5ffc None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:47.209810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c05e9-1361-41f7-b015-d6e0975f5ffc None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:47.210124 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c05e9-1361-41f7-b015-d6e0975f5ffc None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:47.211458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 162/321] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:47.216550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53244f90-70ce-4577-965e-e5600c7e1913 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:47.216832 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53244f90-70ce-4577-965e-e5600c7e1913 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:47.217098 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53244f90-70ce-4577-965e-e5600c7e1913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:47.236667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-53244f90-70ce-4577-965e-e5600c7e1913 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:47.237841 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-53244f90-70ce-4577-965e-e5600c7e1913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:47.302876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 160/322] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:47.458443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c05e9-1361-41f7-b015-d6e0975f5ffc None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:47.473514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea76db4-d550-4826-9aa4-cf19bbf2f7bd None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:47.473854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea76db4-d550-4826-9aa4-cf19bbf2f7bd None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:47.474095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea76db4-d550-4826-9aa4-cf19bbf2f7bd None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:47.475680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 161/323] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:23:47.480546 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f95afd-44e7-4d13-a80e-9b4d88a0c6a4 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:47.480823 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f95afd-44e7-4d13-a80e-9b4d88a0c6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:47.481093 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f95afd-44e7-4d13-a80e-9b4d88a0c6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:47.485350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:47.504316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-40f95afd-44e7-4d13-a80e-9b4d88a0c6a4 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:47.505494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-40f95afd-44e7-4d13-a80e-9b4d88a0c6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:47.566975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c05e9-1361-41f7-b015-d6e0975f5ffc None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:47.576533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 162/324] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:47.579342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f95afd-44e7-4d13-a80e-9b4d88a0c6a4 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:47.602444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f171d1-bdb9-4514-8b21-af52a0f1b51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:47.603137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f171d1-bdb9-4514-8b21-af52a0f1b51f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:47.605764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f171d1-bdb9-4514-8b21-af52a0f1b51f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:47.605764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f171d1-bdb9-4514-8b21-af52a0f1b51f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:47.608444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:47.615051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f171d1-bdb9-4514-8b21-af52a0f1b51f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-88f171d1-bdb9-4514-8b21-af52a0f1b51f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:47.619911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:47.620794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:47.621576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:47.621576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:47.621576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:47.621576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:47.621576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:47.621576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:47.621576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:47.621576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:47.621576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:47.621576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:47.621576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:47.621576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:47.621576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 163/325] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:47.626878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f171d1-bdb9-4514-8b21-af52a0f1b51f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:47.690887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f95afd-44e7-4d13-a80e-9b4d88a0c6a4 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:47.746419 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:47.746571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee9afd-e9a6-4df7-9185-04d63b68fafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:47.746571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee9afd-e9a6-4df7-9185-04d63b68fafb None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:47.746571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee9afd-e9a6-4df7-9185-04d63b68fafb None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:47.746571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee9afd-e9a6-4df7-9185-04d63b68fafb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:47.746888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee9afd-e9a6-4df7-9185-04d63b68fafb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:47.746888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee9afd-e9a6-4df7-9185-04d63b68fafb None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:47.746888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 163/326] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:23:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:47.746888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee9afd-e9a6-4df7-9185-04d63b68fafb None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:47.747150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f171d1-bdb9-4514-8b21-af52a0f1b51f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:47.778541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ec79b-2d06-4bfd-97ab-1d24fb61fb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:47.779143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ec79b-2d06-4bfd-97ab-1d24fb61fb4b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:47.779411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ec79b-2d06-4bfd-97ab-1d24fb61fb4b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:47.779672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ec79b-2d06-4bfd-97ab-1d24fb61fb4b None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:47.780321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ec79b-2d06-4bfd-97ab-1d24fb61fb4b None admin] RBAC: Adding query filter params (name=member) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:47.780604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ec79b-2d06-4bfd-97ab-1d24fb61fb4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:47.782563 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ec79b-2d06-4bfd-97ab-1d24fb61fb4b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:47.795373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 164/327] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:47.796330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ec79b-2d06-4bfd-97ab-1d24fb61fb4b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:47.842733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:47.849160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee9afd-e9a6-4df7-9185-04d63b68fafb None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:47.882667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e98d0b-f6bc-4314-8bc9-067927d7aece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:47.883357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e98d0b-f6bc-4314-8bc9-067927d7aece None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:47.883650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e98d0b-f6bc-4314-8bc9-067927d7aece None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:47.883889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e98d0b-f6bc-4314-8bc9-067927d7aece None admin] PATH_INFO: `/v3/groups` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:47.884504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e98d0b-f6bc-4314-8bc9-067927d7aece None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:47.885986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e98d0b-f6bc-4314-8bc9-067927d7aece None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:47.899083 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ec79b-2d06-4bfd-97ab-1d24fb61fb4b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:47.931234 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795569cf-bcbc-40c5-9455-a98e97fae131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:47.932142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795569cf-bcbc-40c5-9455-a98e97fae131 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:47.932142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795569cf-bcbc-40c5-9455-a98e97fae131 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:47.932466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795569cf-bcbc-40c5-9455-a98e97fae131 None admin] PATH_INFO: `/v3/users/9673ba8598f848c2a6f78d4ac5da7bcf` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:47.933439 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795569cf-bcbc-40c5-9455-a98e97fae131 None admin] RBAC: Authorizing `identity:get_user(user_id=9673ba8598f848c2a6f78d4ac5da7bcf)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:47.934872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795569cf-bcbc-40c5-9455-a98e97fae131 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:47.936410 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 165/328] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:47 2022] GET /identity/v3/users/9673ba8598f848c2a6f78d4ac5da7bcf => generated 293 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:47.937147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795569cf-bcbc-40c5-9455-a98e97fae131 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:47.952158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:47.986229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 164/329] 10.222.0.49 () {60 vars in 1134 bytes} [Fri Sep 16 05:23:47 2022] POST /identity/v3/groups => generated 226 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:23:47.986229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e98d0b-f6bc-4314-8bc9-067927d7aece None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:48.056173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795569cf-bcbc-40c5-9455-a98e97fae131 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:48.095370 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e98d0b-f6bc-4314-8bc9-067927d7aece None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:48.095959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d8a98-2c4b-4db6-85b6-1c2ed5ccc036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:48.096365 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d8a98-2c4b-4db6-85b6-1c2ed5ccc036 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:48.096446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d8a98-2c4b-4db6-85b6-1c2ed5ccc036 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:48.096798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98d8a98-2c4b-4db6-85b6-1c2ed5ccc036 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:48.111494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98d8a98-2c4b-4db6-85b6-1c2ed5ccc036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-d98d8a98-2c4b-4db6-85b6-1c2ed5ccc036 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:48.117666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:48.118731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:48.122604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:48.122604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:48.122604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:48.122604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:48.122604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:48.122604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:48.122604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:48.122604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:48.122604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:48.122604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:48.122604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:48.122604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:48.122604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 166/330] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:48.125187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d8a98-2c4b-4db6-85b6-1c2ed5ccc036 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:48.134914 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345bfd0-eee7-4846-9d80-3bbbedb30cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:48.135543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345bfd0-eee7-4846-9d80-3bbbedb30cca None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:48.135818 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345bfd0-eee7-4846-9d80-3bbbedb30cca None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:48.136092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345bfd0-eee7-4846-9d80-3bbbedb30cca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:48.136838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345bfd0-eee7-4846-9d80-3bbbedb30cca None admin] RBAC: Adding query filter params (role.id=413cc5f8baab440bab827a3547b65f33, user.id=9673ba8598f848c2a6f78d4ac5da7bcf, scope.system=all) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:48.137143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345bfd0-eee7-4846-9d80-3bbbedb30cca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:48.138648 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345bfd0-eee7-4846-9d80-3bbbedb30cca None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:48.151856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 165/331] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Sep 16 05:23:47 2022] GET /identity/v3/role_assignments?user.id=9673ba8598f848c2a6f78d4ac5da7bcf&scope.system=all&role.id=413cc5f8baab440bab827a3547b65f33 => generated 510 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:48.216777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:48.251181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98d8a98-2c4b-4db6-85b6-1c2ed5ccc036 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:48.293304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0043e8-451a-4eae-ae4f-bd8cb34e2780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:48.293992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0043e8-451a-4eae-ae4f-bd8cb34e2780 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:48.294262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0043e8-451a-4eae-ae4f-bd8cb34e2780 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:48.294525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0043e8-451a-4eae-ae4f-bd8cb34e2780 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:48.295199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0043e8-451a-4eae-ae4f-bd8cb34e2780 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:48.295497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0043e8-451a-4eae-ae4f-bd8cb34e2780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:48.296882 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0043e8-451a-4eae-ae4f-bd8cb34e2780 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:48.306910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 167/332] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:48.312316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345bfd0-eee7-4846-9d80-3bbbedb30cca None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:48.334569 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:48.429374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345bfd0-eee7-4846-9d80-3bbbedb30cca None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:48.463349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b837169-fd9e-499d-ae75-739355dda88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:48.463979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b837169-fd9e-499d-ae75-739355dda88f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:48.464251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b837169-fd9e-499d-ae75-739355dda88f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:48.464513 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b837169-fd9e-499d-ae75-739355dda88f None admin] PATH_INFO: `/v3/users/03da65d67cf94c37a9dc8c1589433ca7` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:48.465633 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b837169-fd9e-499d-ae75-739355dda88f None admin] RBAC: Authorizing `identity:get_user(user_id=03da65d67cf94c37a9dc8c1589433ca7)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:48.467176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b837169-fd9e-499d-ae75-739355dda88f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:48.469047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 166/333] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:48 2022] GET /identity/v3/users/03da65d67cf94c37a9dc8c1589433ca7 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:48.475304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0043e8-451a-4eae-ae4f-bd8cb34e2780 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:48.595501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0043e8-451a-4eae-ae4f-bd8cb34e2780 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:48.627105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362fdd63-50a3-4a48-853f-8cf3eb1d0f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:48.627769 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fdd63-50a3-4a48-853f-8cf3eb1d0f86 None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:48.628107 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fdd63-50a3-4a48-853f-8cf3eb1d0f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:48.628381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362fdd63-50a3-4a48-853f-8cf3eb1d0f86 None admin] PATH_INFO: `/v3/system/users/03da65d67cf94c37a9dc8c1589433ca7/roles/c8fdda710e4f416380f2e8d61fa5b71f` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:48.629854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362fdd63-50a3-4a48-853f-8cf3eb1d0f86 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=03da65d67cf94c37a9dc8c1589433ca7, role_id=c8fdda710e4f416380f2e8d61fa5b71f)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:48.631259 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362fdd63-50a3-4a48-853f-8cf3eb1d0f86 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:48.633080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:48.718029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 168/334] 10.222.0.49 () {58 vars in 1334 bytes} [Fri Sep 16 05:23:48 2022] PUT /identity/v3/system/users/03da65d67cf94c37a9dc8c1589433ca7/roles/c8fdda710e4f416380f2e8d61fa5b71f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:23:50.824992 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97208f38-2026-4cfa-95ba-9379a9ebda0c None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:50.825529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97208f38-2026-4cfa-95ba-9379a9ebda0c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:50.825529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97208f38-2026-4cfa-95ba-9379a9ebda0c None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:50.827108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 167/335] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:50.833301 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae9589-4726-43eb-9926-ec682cee4deb None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:50.833723 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae9589-4726-43eb-9926-ec682cee4deb None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:50.833943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae9589-4726-43eb-9926-ec682cee4deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:50.858818 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-bbae9589-4726-43eb-9926-ec682cee4deb None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:50.860042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-bbae9589-4726-43eb-9926-ec682cee4deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:50.956498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 169/336] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:51.071022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aabc1e-f48d-45a7-a1ba-00c1bc1ce73e None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:51.071656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aabc1e-f48d-45a7-a1ba-00c1bc1ce73e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:51.072149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aabc1e-f48d-45a7-a1ba-00c1bc1ce73e None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:51.073990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 168/337] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:51.083357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a19558-9ec0-48a3-807b-1fe7187dc9bb None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:51.083920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a19558-9ec0-48a3-807b-1fe7187dc9bb None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:51.084392 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a19558-9ec0-48a3-807b-1fe7187dc9bb None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:51.086205 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 169/338] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:51.087874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea22016-7cef-4586-bc16-d79dfe04e978 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:51.088392 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea22016-7cef-4586-bc16-d79dfe04e978 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:51.088855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea22016-7cef-4586-bc16-d79dfe04e978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:51.090835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc784e-efcd-436d-a019-5a4a80df4edb None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:51.091337 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc784e-efcd-436d-a019-5a4a80df4edb None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:51.091816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc784e-efcd-436d-a019-5a4a80df4edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:51.127374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-2ea22016-7cef-4586-bc16-d79dfe04e978 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:51.128614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-bfcc784e-efcd-436d-a019-5a4a80df4edb None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:51.130049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2ea22016-7cef-4586-bc16-d79dfe04e978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:51.131403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-bfcc784e-efcd-436d-a019-5a4a80df4edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:51.265904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 170/339] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:51.268646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 170/340] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:51.455060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc784e-efcd-436d-a019-5a4a80df4edb None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:51.470604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:51.561928 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc784e-efcd-436d-a019-5a4a80df4edb None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:51.600965 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a254d2b-89a1-46a9-ba43-86e51df9a78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:51.604829 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a254d2b-89a1-46a9-ba43-86e51df9a78a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:51.604829 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a254d2b-89a1-46a9-ba43-86e51df9a78a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:51.604829 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a254d2b-89a1-46a9-ba43-86e51df9a78a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:51.614547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea22016-7cef-4586-bc16-d79dfe04e978 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:51.618069 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a254d2b-89a1-46a9-ba43-86e51df9a78a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-7a254d2b-89a1-46a9-ba43-86e51df9a78a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:51.623175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:51.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:51.624866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:51.624866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:51.624866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:51.624866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:51.624866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:51.624866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:51.624866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:51.624866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:51.624866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:51.624866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:51.624866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:51.624866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:51.624866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 171/341] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:23:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:51.631470 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a254d2b-89a1-46a9-ba43-86e51df9a78a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:51.735964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:51.757565 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:51.814191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a254d2b-89a1-46a9-ba43-86e51df9a78a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:51.818468 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea22016-7cef-4586-bc16-d79dfe04e978 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:51.883101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96e28b-bb58-4358-a4b1-6c6d79db24fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:51.883787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96e28b-bb58-4358-a4b1-6c6d79db24fd None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:51.883959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96e28b-bb58-4358-a4b1-6c6d79db24fd None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:51.884292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96e28b-bb58-4358-a4b1-6c6d79db24fd None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:51.884961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96e28b-bb58-4358-a4b1-6c6d79db24fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:51.885255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96e28b-bb58-4358-a4b1-6c6d79db24fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:51.886667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea96e28b-bb58-4358-a4b1-6c6d79db24fd None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:51.889590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a04ca-c00b-4956-a174-0eeadd50ec56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:51.890248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a04ca-c00b-4956-a174-0eeadd50ec56 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:51.890514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a04ca-c00b-4956-a174-0eeadd50ec56 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:51.890769 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a04ca-c00b-4956-a174-0eeadd50ec56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:51.905856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 172/342] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:23:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:51.914514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96e28b-bb58-4358-a4b1-6c6d79db24fd None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:51.916765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a04ca-c00b-4956-a174-0eeadd50ec56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-894a04ca-c00b-4956-a174-0eeadd50ec56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:51.919864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:51.920764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 171/343] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:23:51.921559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a04ca-c00b-4956-a174-0eeadd50ec56 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:52.087138 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96e28b-bb58-4358-a4b1-6c6d79db24fd None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:52.119010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a04ca-c00b-4956-a174-0eeadd50ec56 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:52.132910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cf4bc-a0e5-457d-bb9e-681062cadaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:52.133545 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cf4bc-a0e5-457d-bb9e-681062cadaec None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:52.133871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cf4bc-a0e5-457d-bb9e-681062cadaec None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:52.134108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5cf4bc-a0e5-457d-bb9e-681062cadaec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:52.138421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:52.138540 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:52.152427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5cf4bc-a0e5-457d-bb9e-681062cadaec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:52.157750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e798c8-f7d8-4b5e-9f26-6c9485a43ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-ee5cf4bc-a0e5-457d-bb9e-681062cadaec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:52.157922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:52.158838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:52.159659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:52.159659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:52.159659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:52.159659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:52.159659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:52.159659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:52.159659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:52.159659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:52.159659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:52.159659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:52.159659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:52.159659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:52.160278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e798c8-f7d8-4b5e-9f26-6c9485a43ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:52.160278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e798c8-f7d8-4b5e-9f26-6c9485a43ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:52.160278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e798c8-f7d8-4b5e-9f26-6c9485a43ea8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:52.160457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 173/344] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:52.160457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cf4bc-a0e5-457d-bb9e-681062cadaec None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:52.238541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e798c8-f7d8-4b5e-9f26-6c9485a43ea8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:52.240000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e798c8-f7d8-4b5e-9f26-6c9485a43ea8 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:52.272402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5cf4bc-a0e5-457d-bb9e-681062cadaec None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:52.306802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91001104-cd47-410a-b4d5-207cbeaf02c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:52.307404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91001104-cd47-410a-b4d5-207cbeaf02c9 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:52.307674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91001104-cd47-410a-b4d5-207cbeaf02c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:52.307936 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91001104-cd47-410a-b4d5-207cbeaf02c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:52.308592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91001104-cd47-410a-b4d5-207cbeaf02c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:52.308918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91001104-cd47-410a-b4d5-207cbeaf02c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:52.310294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91001104-cd47-410a-b4d5-207cbeaf02c9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:52.319539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 174/345] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:52.320799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615a174-0093-4cbe-9a19-108295702f10 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:52.321037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615a174-0093-4cbe-9a19-108295702f10 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:52.321300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615a174-0093-4cbe-9a19-108295702f10 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:52.322765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 175/346] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:52.323456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615a174-0093-4cbe-9a19-108295702f10 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:52.348470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 172/347] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:52.349083 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e798c8-f7d8-4b5e-9f26-6c9485a43ea8 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:52.373043 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:52.395603 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:52.456679 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615a174-0093-4cbe-9a19-108295702f10 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:52.478156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e798c8-f7d8-4b5e-9f26-6c9485a43ea8 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:52.507708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d74d67-8f59-4016-9561-8dc180c9537a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:52.508313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d74d67-8f59-4016-9561-8dc180c9537a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:52.508583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d74d67-8f59-4016-9561-8dc180c9537a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:52.508847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d74d67-8f59-4016-9561-8dc180c9537a None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:52.509560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d74d67-8f59-4016-9561-8dc180c9537a None admin] RBAC: Adding query filter params (name=member) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:52.509792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d74d67-8f59-4016-9561-8dc180c9537a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:52.511216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d74d67-8f59-4016-9561-8dc180c9537a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:52.524001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 176/348] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:52.525792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539cff-539c-406c-9827-0dc16a28199d None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:52.526497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539cff-539c-406c-9827-0dc16a28199d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:52.526978 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7539cff-539c-406c-9827-0dc16a28199d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:52.531050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcc210-9e55-4969-8df8-a9bb62630727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:52.531805 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcc210-9e55-4969-8df8-a9bb62630727 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:52.532216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcc210-9e55-4969-8df8-a9bb62630727 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:52.532514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcc210-9e55-4969-8df8-a9bb62630727 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:52.548170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a7539cff-539c-406c-9827-0dc16a28199d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:52.549373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a7539cff-539c-406c-9827-0dc16a28199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:52.620264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdcc210-9e55-4969-8df8-a9bb62630727 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:52.621815 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdcc210-9e55-4969-8df8-a9bb62630727 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:52.624831 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 177/349] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:52.627846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7007c9d-73af-479e-bcc6-f5c6015010b7 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:52.627846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7007c9d-73af-479e-bcc6-f5c6015010b7 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:52.627846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7007c9d-73af-479e-bcc6-f5c6015010b7 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:52.628717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 178/350] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:52.629412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7007c9d-73af-479e-bcc6-f5c6015010b7 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:52.645396 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:52.689791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:52.736354 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 173/351] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:52.737423 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcc210-9e55-4969-8df8-a9bb62630727 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:52.745235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7007c9d-73af-479e-bcc6-f5c6015010b7 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:52.798025 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d30a3-6124-42aa-9e1c-5ed8a9148e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:52.798925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d30a3-6124-42aa-9e1c-5ed8a9148e6b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:52.799270 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d30a3-6124-42aa-9e1c-5ed8a9148e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:52.799614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d30a3-6124-42aa-9e1c-5ed8a9148e6b None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:52.800927 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d30a3-6124-42aa-9e1c-5ed8a9148e6b None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:52.802523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d30a3-6124-42aa-9e1c-5ed8a9148e6b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:52.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 179/352] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:52 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:52.809313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b105e-45f0-4454-b868-d5edd0c9d147 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:52.809447 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b105e-45f0-4454-b868-d5edd0c9d147 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:52.809712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b105e-45f0-4454-b868-d5edd0c9d147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:52.829707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-222b105e-45f0-4454-b868-d5edd0c9d147 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:52.831046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-222b105e-45f0-4454-b868-d5edd0c9d147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:52.846981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcc210-9e55-4969-8df8-a9bb62630727 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:52.865270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:52.884027 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad0dd0f-3968-49fd-9515-aad20ffa7e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:52.884960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0dd0f-3968-49fd-9515-aad20ffa7e2e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:52.885439 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0dd0f-3968-49fd-9515-aad20ffa7e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:52.885933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0dd0f-3968-49fd-9515-aad20ffa7e2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:52.904592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 180/353] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:52.905530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b105e-45f0-4454-b868-d5edd0c9d147 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:52.966602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad0dd0f-3968-49fd-9515-aad20ffa7e2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:52.967977 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:52.968088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad0dd0f-3968-49fd-9515-aad20ffa7e2e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:53.050231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b105e-45f0-4454-b868-d5edd0c9d147 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:53.100101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411adbc-d38c-4fd5-86b7-23229a248322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:53.100767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411adbc-d38c-4fd5-86b7-23229a248322 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:53.101204 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411adbc-d38c-4fd5-86b7-23229a248322 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:53.101615 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411adbc-d38c-4fd5-86b7-23229a248322 None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:53.102920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411adbc-d38c-4fd5-86b7-23229a248322 None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:53.103554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 174/354] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:53.104394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad0dd0f-3968-49fd-9515-aad20ffa7e2e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:53.105097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411adbc-d38c-4fd5-86b7-23229a248322 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:53.107220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 181/355] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:52 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:53.108164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411adbc-d38c-4fd5-86b7-23229a248322 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:53.129591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:53.237430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad0dd0f-3968-49fd-9515-aad20ffa7e2e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:53.254041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411adbc-d38c-4fd5-86b7-23229a248322 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:53.271771 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:53.280724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca01bc-aedf-417b-a402-2f6583c9c3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:53.281361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca01bc-aedf-417b-a402-2f6583c9c3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:53.282974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca01bc-aedf-417b-a402-2f6583c9c3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:53.290074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca01bc-aedf-417b-a402-2f6583c9c3d0 None admin] PATH_INFO: `/v3/groups/b8324c97af7b4541b768ec0f4a9ae903` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:53.290074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca01bc-aedf-417b-a402-2f6583c9c3d0 None admin] RBAC: Authorizing `identity:get_group(group_id=b8324c97af7b4541b768ec0f4a9ae903)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:53.290742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca01bc-aedf-417b-a402-2f6583c9c3d0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:53.292510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 175/356] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:23:53 2022] GET /identity/v3/groups/b8324c97af7b4541b768ec0f4a9ae903 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:53.293507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392eae2-d4c9-4b5a-8a2b-9eebced83bdb None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:53.293779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392eae2-d4c9-4b5a-8a2b-9eebced83bdb None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:53.294072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392eae2-d4c9-4b5a-8a2b-9eebced83bdb None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:53.295258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 176/357] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:53.295925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392eae2-d4c9-4b5a-8a2b-9eebced83bdb None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:53.300547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a76d2-cf6a-4914-b8bc-88431d1bdf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:53.301187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a76d2-cf6a-4914-b8bc-88431d1bdf99 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:53.301620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a76d2-cf6a-4914-b8bc-88431d1bdf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:53.301901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a76d2-cf6a-4914-b8bc-88431d1bdf99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:53.314600 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a76d2-cf6a-4914-b8bc-88431d1bdf99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-5b0a76d2-cf6a-4914-b8bc-88431d1bdf99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:53.319473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:53.320361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:53.321158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:53.321158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:53.321158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:53.321158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:53.321158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:53.321158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:53.321158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:53.321158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:53.321158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:53.321158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:53.321158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:53.321158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:53.321158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 182/358] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:23:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:53.322377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a76d2-cf6a-4914-b8bc-88431d1bdf99 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:53.398573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:53.426019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392eae2-d4c9-4b5a-8a2b-9eebced83bdb None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:53.449612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a76d2-cf6a-4914-b8bc-88431d1bdf99 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:53.466264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cbc44-ef30-4a33-a409-195bce6b4e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:53.467208 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cbc44-ef30-4a33-a409-195bce6b4e0d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:53.467646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cbc44-ef30-4a33-a409-195bce6b4e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:53.468071 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cbc44-ef30-4a33-a409-195bce6b4e0d None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:53.469390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cbc44-ef30-4a33-a409-195bce6b4e0d None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:53.471114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cbc44-ef30-4a33-a409-195bce6b4e0d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:53.472763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 177/359] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:53 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:53.473529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cbc44-ef30-4a33-a409-195bce6b4e0d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:53.488485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32767b-ac09-441e-9d4a-bdd5bec3bf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:53.489043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32767b-ac09-441e-9d4a-bdd5bec3bf56 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:53.489315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32767b-ac09-441e-9d4a-bdd5bec3bf56 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:53.489583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32767b-ac09-441e-9d4a-bdd5bec3bf56 None admin] PATH_INFO: `/v3/groups/b8324c97af7b4541b768ec0f4a9ae903` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:53.490967 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32767b-ac09-441e-9d4a-bdd5bec3bf56 None admin] RBAC: Authorizing `identity:get_group(group_id=b8324c97af7b4541b768ec0f4a9ae903)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:53.492403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32767b-ac09-441e-9d4a-bdd5bec3bf56 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:53.494015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 183/360] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:23:53 2022] GET /identity/v3/groups/b8324c97af7b4541b768ec0f4a9ae903 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:53.498163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32767b-ac09-441e-9d4a-bdd5bec3bf56 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:53.550067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:53.593636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cbc44-ef30-4a33-a409-195bce6b4e0d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:53.628282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32767b-ac09-441e-9d4a-bdd5bec3bf56 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:53.630731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b6de49-81d3-4598-b809-92fab7c154c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:53.631569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6de49-81d3-4598-b809-92fab7c154c7 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:53.631921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6de49-81d3-4598-b809-92fab7c154c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:53.632173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b6de49-81d3-4598-b809-92fab7c154c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:53.645492 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b6de49-81d3-4598-b809-92fab7c154c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-44b6de49-81d3-4598-b809-92fab7c154c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:53.650828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:53.651779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:53.652720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:53.652720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:53.652720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:53.652720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:53.652720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:53.652720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:53.652720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:53.652720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:53.652720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:53.652720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:53.652720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 16 05:23:53.652720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:53.652720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 178/361] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:53.654556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69561058-b3e4-4817-9fb5-444505cf798f None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:53.654556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69561058-b3e4-4817-9fb5-444505cf798f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:53.654556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69561058-b3e4-4817-9fb5-444505cf798f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:53.664347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47887c-10b3-4b12-9148-650c5df56640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:53.664951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47887c-10b3-4b12-9148-650c5df56640 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:53.665221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47887c-10b3-4b12-9148-650c5df56640 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:53.665483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47887c-10b3-4b12-9148-650c5df56640 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:53.666238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47887c-10b3-4b12-9148-650c5df56640 None admin] RBAC: Adding query filter params (group.id=b8324c97af7b4541b768ec0f4a9ae903, role.id=93f699f03f0f41498415ab9533bdfcb4, scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:53.666729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47887c-10b3-4b12-9148-650c5df56640 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:53.668171 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c47887c-10b3-4b12-9148-650c5df56640 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:53.674652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-69561058-b3e4-4817-9fb5-444505cf798f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:53.674820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 184/362] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 16 05:23:53 2022] GET /identity/v3/role_assignments?group.id=b8324c97af7b4541b768ec0f4a9ae903&scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e&role.id=93f699f03f0f41498415ab9533bdfcb4 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:53.675580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47887c-10b3-4b12-9148-650c5df56640 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:53.676384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-69561058-b3e4-4817-9fb5-444505cf798f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:53.679693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:53.773956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 179/363] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:53.773956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69561058-b3e4-4817-9fb5-444505cf798f None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:53.814374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47887c-10b3-4b12-9148-650c5df56640 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:53.823501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:53.857244 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c22427-0763-4245-9adb-ece73a23f5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:53.859225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c22427-0763-4245-9adb-ece73a23f5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:53.859743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c22427-0763-4245-9adb-ece73a23f5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:53.860487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c22427-0763-4245-9adb-ece73a23f5b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:53.861592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c22427-0763-4245-9adb-ece73a23f5b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:53.862527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c22427-0763-4245-9adb-ece73a23f5b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:53.864153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c22427-0763-4245-9adb-ece73a23f5b8 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:53.875890 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 185/364] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:23:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:53.877024 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c22427-0763-4245-9adb-ece73a23f5b8 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:53.932751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69561058-b3e4-4817-9fb5-444505cf798f None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:53.972712 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5ee4d-ce36-40af-9591-efc570a37ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:53.974458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5ee4d-ce36-40af-9591-efc570a37ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:53.974883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5ee4d-ce36-40af-9591-efc570a37ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:53.975296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5ee4d-ce36-40af-9591-efc570a37ab2 None admin] PATH_INFO: `/v3/groups/b8324c97af7b4541b768ec0f4a9ae903` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:53.977383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5ee4d-ce36-40af-9591-efc570a37ab2 None admin] RBAC: Authorizing `identity:get_group(group_id=b8324c97af7b4541b768ec0f4a9ae903)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:53.979360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5ee4d-ce36-40af-9591-efc570a37ab2 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:53.981391 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 180/365] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:23:53 2022] GET /identity/v3/groups/b8324c97af7b4541b768ec0f4a9ae903 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:53.982366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5ee4d-ce36-40af-9591-efc570a37ab2 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:53.999084 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:54.008853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c22427-0763-4245-9adb-ece73a23f5b8 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:54.065154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70458d4-25d2-4dee-b400-0a4d357467d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:54.066535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70458d4-25d2-4dee-b400-0a4d357467d5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:54.067017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70458d4-25d2-4dee-b400-0a4d357467d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:54.067511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70458d4-25d2-4dee-b400-0a4d357467d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:54.068453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70458d4-25d2-4dee-b400-0a4d357467d5 None admin] RBAC: Adding query filter params (group.id=b8324c97af7b4541b768ec0f4a9ae903, role.id=413cc5f8baab440bab827a3547b65f33, scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:54.069446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70458d4-25d2-4dee-b400-0a4d357467d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:54.071563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70458d4-25d2-4dee-b400-0a4d357467d5 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:54.079925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 186/366] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 16 05:23:53 2022] GET /identity/v3/role_assignments?group.id=b8324c97af7b4541b768ec0f4a9ae903&scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e&role.id=413cc5f8baab440bab827a3547b65f33 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:54.081034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70458d4-25d2-4dee-b400-0a4d357467d5 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:54.123416 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:54.145286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5ee4d-ce36-40af-9591-efc570a37ab2 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:54.192391 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a63e83-e12c-4359-bc01-bbf78528a2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:54.193473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a63e83-e12c-4359-bc01-bbf78528a2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:54.193987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a63e83-e12c-4359-bc01-bbf78528a2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:54.194487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a63e83-e12c-4359-bc01-bbf78528a2a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:54.195374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a63e83-e12c-4359-bc01-bbf78528a2a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:54.196178 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a63e83-e12c-4359-bc01-bbf78528a2a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:54.198757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a63e83-e12c-4359-bc01-bbf78528a2a5 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:54.206640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70458d4-25d2-4dee-b400-0a4d357467d5 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:54.210635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 181/367] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:54.211658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a63e83-e12c-4359-bc01-bbf78528a2a5 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:54.248802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b18740-459e-4d30-9545-f0ae4e85a657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:54.249814 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b18740-459e-4d30-9545-f0ae4e85a657 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:54.250342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b18740-459e-4d30-9545-f0ae4e85a657 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:54.250853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b18740-459e-4d30-9545-f0ae4e85a657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:54.307459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:54.337743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b18740-459e-4d30-9545-f0ae4e85a657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:54.340131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b18740-459e-4d30-9545-f0ae4e85a657 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:54.343746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a63e83-e12c-4359-bc01-bbf78528a2a5 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:54.389507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:54.392982 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf55d98-7d48-48f9-85e8-6d91a1ef8147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:54.393866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf55d98-7d48-48f9-85e8-6d91a1ef8147 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:54.394317 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf55d98-7d48-48f9-85e8-6d91a1ef8147 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:54.394755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf55d98-7d48-48f9-85e8-6d91a1ef8147 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:54.395725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf55d98-7d48-48f9-85e8-6d91a1ef8147 None admin] RBAC: Adding query filter params (group.id=b8324c97af7b4541b768ec0f4a9ae903, role.id=413cc5f8baab440bab827a3547b65f33, scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:54.396283 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf55d98-7d48-48f9-85e8-6d91a1ef8147 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:54.398158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf55d98-7d48-48f9-85e8-6d91a1ef8147 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:54.406545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 182/368] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 16 05:23:54 2022] GET /identity/v3/role_assignments?group.id=b8324c97af7b4541b768ec0f4a9ae903&scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe&role.id=413cc5f8baab440bab827a3547b65f33 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:54.406545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf55d98-7d48-48f9-85e8-6d91a1ef8147 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:54.526758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 187/369] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:54.528807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b18740-459e-4d30-9545-f0ae4e85a657 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:54.624980 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf55d98-7d48-48f9-85e8-6d91a1ef8147 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:54.677294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13547cb4-672d-4f97-9ed9-c8759a626b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:54.677964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13547cb4-672d-4f97-9ed9-c8759a626b90 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:54.678244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13547cb4-672d-4f97-9ed9-c8759a626b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:54.678516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13547cb4-672d-4f97-9ed9-c8759a626b90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:54.683927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:54.699490 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13547cb4-672d-4f97-9ed9-c8759a626b90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-13547cb4-672d-4f97-9ed9-c8759a626b90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:54.714021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:54.716004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:54.716912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:54.716912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:54.716912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:54.716912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:54.716912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:54.716912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:54.716912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:54.716912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:54.716912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:54.716912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:54.716912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:23:54.716912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:54.719910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 183/370] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:23:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:54.721688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13547cb4-672d-4f97-9ed9-c8759a626b90 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:54.734506 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b18740-459e-4d30-9545-f0ae4e85a657 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:54.811027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:54.828028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395fe64-edb1-490b-bbe0-60ac0bb90278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:54.829533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395fe64-edb1-490b-bbe0-60ac0bb90278 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:54.829844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395fe64-edb1-490b-bbe0-60ac0bb90278 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:54.831027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1395fe64-edb1-490b-bbe0-60ac0bb90278 None admin] PATH_INFO: `/v3/users/03da65d67cf94c37a9dc8c1589433ca7` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:54.836994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395fe64-edb1-490b-bbe0-60ac0bb90278 None admin] RBAC: Authorizing `identity:get_user(user_id=03da65d67cf94c37a9dc8c1589433ca7)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:54.840819 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1395fe64-edb1-490b-bbe0-60ac0bb90278 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:54.842893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 188/371] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:23:54 2022] GET /identity/v3/users/03da65d67cf94c37a9dc8c1589433ca7 => generated 293 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:54.843830 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395fe64-edb1-490b-bbe0-60ac0bb90278 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:54.894024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13547cb4-672d-4f97-9ed9-c8759a626b90 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:54.991568 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83646646-09cd-4720-8d49-da25af05f3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:54.992599 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83646646-09cd-4720-8d49-da25af05f3dc None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:54.993320 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83646646-09cd-4720-8d49-da25af05f3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:54.993597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83646646-09cd-4720-8d49-da25af05f3dc None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:54.995501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83646646-09cd-4720-8d49-da25af05f3dc None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:54.996968 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83646646-09cd-4720-8d49-da25af05f3dc None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:54.998960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 184/372] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:54 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:54.999701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83646646-09cd-4720-8d49-da25af05f3dc None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:55.071127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1395fe64-edb1-490b-bbe0-60ac0bb90278 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:55.130781 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:55.149716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8499f-37fe-4cae-8035-7e5cd3f266ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:55.149716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8499f-37fe-4cae-8035-7e5cd3f266ca None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:55.149716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8499f-37fe-4cae-8035-7e5cd3f266ca None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:55.149716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8499f-37fe-4cae-8035-7e5cd3f266ca None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:55.150249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8499f-37fe-4cae-8035-7e5cd3f266ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:55.150249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8499f-37fe-4cae-8035-7e5cd3f266ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:55.158841 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8499f-37fe-4cae-8035-7e5cd3f266ca None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:55.177283 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 189/373] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:55.179319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8499f-37fe-4cae-8035-7e5cd3f266ca None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:55.223521 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83646646-09cd-4720-8d49-da25af05f3dc None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:55.292933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:55.297984 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a9d8c-0586-475d-98df-20c539d54b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:55.298754 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a9d8c-0586-475d-98df-20c539d54b02 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:55.299031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a9d8c-0586-475d-98df-20c539d54b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:55.299298 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a9d8c-0586-475d-98df-20c539d54b02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:55.300026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a9d8c-0586-475d-98df-20c539d54b02 None admin] RBAC: Adding query filter params (role.id=c8fdda710e4f416380f2e8d61fa5b71f, user.id=03da65d67cf94c37a9dc8c1589433ca7, scope.system=all) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:55.300328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a9d8c-0586-475d-98df-20c539d54b02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:55.301795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a9d8c-0586-475d-98df-20c539d54b02 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:55.332641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 185/374] 10.222.0.49 () {56 vars in 1308 bytes} [Fri Sep 16 05:23:54 2022] GET /identity/v3/role_assignments?user.id=03da65d67cf94c37a9dc8c1589433ca7&scope.system=all&role.id=c8fdda710e4f416380f2e8d61fa5b71f => generated 510 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:55.333260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a9d8c-0586-475d-98df-20c539d54b02 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:55.383961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8499f-37fe-4cae-8035-7e5cd3f266ca None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:55.479387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752b4cc-4518-4c79-a209-cfb137ca3c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:55.480294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752b4cc-4518-4c79-a209-cfb137ca3c43 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:55.480718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752b4cc-4518-4c79-a209-cfb137ca3c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:55.481131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752b4cc-4518-4c79-a209-cfb137ca3c43 None admin] PATH_INFO: `/v3/groups/b8324c97af7b4541b768ec0f4a9ae903` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:55.482726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752b4cc-4518-4c79-a209-cfb137ca3c43 None admin] RBAC: Authorizing `identity:get_group(group_id=b8324c97af7b4541b768ec0f4a9ae903)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:55.484466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752b4cc-4518-4c79-a209-cfb137ca3c43 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:55.487827 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 190/375] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:23:55 2022] GET /identity/v3/groups/b8324c97af7b4541b768ec0f4a9ae903 => generated 226 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:55.499164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752b4cc-4518-4c79-a209-cfb137ca3c43 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:55.570130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a9d8c-0586-475d-98df-20c539d54b02 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:55.618209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:55.662158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d871e-bf17-42ad-98d6-9c2c5a18e00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:55.662777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d871e-bf17-42ad-98d6-9c2c5a18e00a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:55.663045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d871e-bf17-42ad-98d6-9c2c5a18e00a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:55.663308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d871e-bf17-42ad-98d6-9c2c5a18e00a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:55.728880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752b4cc-4518-4c79-a209-cfb137ca3c43 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:55.797384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:55.805217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d871e-bf17-42ad-98d6-9c2c5a18e00a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:55.809748 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d871e-bf17-42ad-98d6-9c2c5a18e00a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:55.811717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ad345-5602-4519-9f7c-a044730eaeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:55.812328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ad345-5602-4519-9f7c-a044730eaeee None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:55.812599 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ad345-5602-4519-9f7c-a044730eaeee None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:55.812902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ad345-5602-4519-9f7c-a044730eaeee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:55.813843 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ad345-5602-4519-9f7c-a044730eaeee None admin] RBAC: Adding query filter params (group.id=b8324c97af7b4541b768ec0f4a9ae903, role.id=93f699f03f0f41498415ab9533bdfcb4, scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:55.814148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ad345-5602-4519-9f7c-a044730eaeee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:55.815613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ad345-5602-4519-9f7c-a044730eaeee None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:55.829072 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 191/376] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 16 05:23:55 2022] GET /identity/v3/role_assignments?group.id=b8324c97af7b4541b768ec0f4a9ae903&scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe&role.id=93f699f03f0f41498415ab9533bdfcb4 => generated 260 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:55.977353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 186/377] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:23:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 644 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:23:55.983894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ad345-5602-4519-9f7c-a044730eaeee None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:56.128484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ad345-5602-4519-9f7c-a044730eaeee None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:56.168205 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c69f15-35c5-42f9-bfdc-4e22002f0754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:56.168832 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c69f15-35c5-42f9-bfdc-4e22002f0754 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:56.169118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c69f15-35c5-42f9-bfdc-4e22002f0754 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:56.169377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c69f15-35c5-42f9-bfdc-4e22002f0754 None admin] PATH_INFO: `/v3/projects/2c317c5acb304c5d983c73b9ed1b6802` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:56.174741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c69f15-35c5-42f9-bfdc-4e22002f0754 None admin] RBAC: Authorizing `identity:get_project(project_id=2c317c5acb304c5d983c73b9ed1b6802)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:56.176266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c69f15-35c5-42f9-bfdc-4e22002f0754 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:56.178293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 192/378] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:55 2022] GET /identity/v3/projects/2c317c5acb304c5d983c73b9ed1b6802 => generated 344 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:56.184012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d871e-bf17-42ad-98d6-9c2c5a18e00a None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:56.207206 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:56.301455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d871e-bf17-42ad-98d6-9c2c5a18e00a None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:56.340622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695ada4-715e-450f-b291-6426aec8fe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:56.341248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695ada4-715e-450f-b291-6426aec8fe3d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:56.341517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695ada4-715e-450f-b291-6426aec8fe3d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:56.341787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695ada4-715e-450f-b291-6426aec8fe3d None admin] PATH_INFO: `/v3/groups/d6bee4d7fb124734a8168a39e29d938d` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:56.347624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695ada4-715e-450f-b291-6426aec8fe3d None admin] RBAC: Authorizing `identity:get_group(group_id=d6bee4d7fb124734a8168a39e29d938d)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:56.349113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695ada4-715e-450f-b291-6426aec8fe3d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:56.351112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 187/379] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:23:56 2022] GET /identity/v3/groups/d6bee4d7fb124734a8168a39e29d938d => generated 229 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:56.358918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c69f15-35c5-42f9-bfdc-4e22002f0754 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:56.429678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:56.475529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c69f15-35c5-42f9-bfdc-4e22002f0754 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:56.517377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be146d5-cc35-4c17-9903-3309798c0f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:56.518022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be146d5-cc35-4c17-9903-3309798c0f16 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:56.518297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be146d5-cc35-4c17-9903-3309798c0f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:56.518562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be146d5-cc35-4c17-9903-3309798c0f16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:56.519303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be146d5-cc35-4c17-9903-3309798c0f16 None admin] RBAC: Adding query filter params (group.id=d6bee4d7fb124734a8168a39e29d938d, role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=2c317c5acb304c5d983c73b9ed1b6802) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:56.519610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be146d5-cc35-4c17-9903-3309798c0f16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:56.521076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be146d5-cc35-4c17-9903-3309798c0f16 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:56.527458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 193/380] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 16 05:23:56 2022] GET /identity/v3/role_assignments?group.id=d6bee4d7fb124734a8168a39e29d938d&scope.project.id=2c317c5acb304c5d983c73b9ed1b6802&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:56.664472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ba0c8-d41c-4217-ba40-1414ba3381ae None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:56.664472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ba0c8-d41c-4217-ba40-1414ba3381ae None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:56.664472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ba0c8-d41c-4217-ba40-1414ba3381ae None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:56.665270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 188/381] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:56.671220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c67ab-ca98-4686-a353-ef30c6953cde None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:56.671755 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c67ab-ca98-4686-a353-ef30c6953cde None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:56.672320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c67ab-ca98-4686-a353-ef30c6953cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:56.695815 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-185c67ab-ca98-4686-a353-ef30c6953cde None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:56.697356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-185c67ab-ca98-4686-a353-ef30c6953cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:56.809182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 194/382] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:57.128814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ba0c8-d41c-4217-ba40-1414ba3381ae None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:57.231692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:57.256961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ba0c8-d41c-4217-ba40-1414ba3381ae None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:57.296452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce3658-61b2-49a9-907f-82bae99f2c57 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:57.298056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce3658-61b2-49a9-907f-82bae99f2c57 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:57.298631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce3658-61b2-49a9-907f-82bae99f2c57 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:57.298980 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ffebc-39c2-4500-8766-1dd791bc4ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:57.299603 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ffebc-39c2-4500-8766-1dd791bc4ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:57.299873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ffebc-39c2-4500-8766-1dd791bc4ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:57.300138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ffebc-39c2-4500-8766-1dd791bc4ad2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:57.301487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 195/383] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:23:57.306472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f25cb-d905-485d-9aa4-d6dab24ec448 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:57.307000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f25cb-d905-485d-9aa4-d6dab24ec448 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:57.307468 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f25cb-d905-485d-9aa4-d6dab24ec448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:57.314316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ffebc-39c2-4500-8766-1dd791bc4ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-271ffebc-39c2-4500-8766-1dd791bc4ad2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:57.319287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:57.320209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:57.321053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:57.321053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:57.321053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:57.321053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:57.321053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:57.321053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:57.321053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:57.321053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:57.321053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:57.321053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:57.321053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:57.321053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:23:57.321053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 189/384] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:23:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:57.326761 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ffebc-39c2-4500-8766-1dd791bc4ad2 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:57.330632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-293f25cb-d905-485d-9aa4-d6dab24ec448 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:57.332180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-293f25cb-d905-485d-9aa4-d6dab24ec448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:57.386581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:57.412469 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 196/385] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:57.434977 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ffebc-39c2-4500-8766-1dd791bc4ad2 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:57.477640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4503919-c188-402c-aaac-16007c13046f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:57.478318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4503919-c188-402c-aaac-16007c13046f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:57.478589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4503919-c188-402c-aaac-16007c13046f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:57.478857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4503919-c188-402c-aaac-16007c13046f None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:57.479553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4503919-c188-402c-aaac-16007c13046f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:57.479855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4503919-c188-402c-aaac-16007c13046f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:57.481212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4503919-c188-402c-aaac-16007c13046f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:57.495345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 190/386] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:23:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:57.501874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f25cb-d905-485d-9aa4-d6dab24ec448 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:57.639728 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f25cb-d905-485d-9aa4-d6dab24ec448 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:57.681910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc5d0f-760d-4ebe-a14f-6e29cf3fcbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:57.682587 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc5d0f-760d-4ebe-a14f-6e29cf3fcbeb None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:57.683006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc5d0f-760d-4ebe-a14f-6e29cf3fcbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:57.683282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc5d0f-760d-4ebe-a14f-6e29cf3fcbeb None admin] PATH_INFO: `/v3/groups/b8324c97af7b4541b768ec0f4a9ae903` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:57.686818 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc5d0f-760d-4ebe-a14f-6e29cf3fcbeb None admin] RBAC: Authorizing `identity:get_group(group_id=b8324c97af7b4541b768ec0f4a9ae903)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:57.689049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc5d0f-760d-4ebe-a14f-6e29cf3fcbeb None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:57.690816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 197/387] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:23:57 2022] GET /identity/v3/groups/b8324c97af7b4541b768ec0f4a9ae903 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:57.696210 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4503919-c188-402c-aaac-16007c13046f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:57.705715 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53fd6c-165e-4d1a-9011-54c0aaf90aab None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:57.706241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53fd6c-165e-4d1a-9011-54c0aaf90aab None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:57.706661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53fd6c-165e-4d1a-9011-54c0aaf90aab None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:57.707252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:57.708827 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 198/388] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:23:57.713903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4af22-c583-41cb-ad2f-6ae2b55a2f37 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:57.717706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4af22-c583-41cb-ad2f-6ae2b55a2f37 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:57.717706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac4af22-c583-41cb-ad2f-6ae2b55a2f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:57.742421 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-eac4af22-c583-41cb-ad2f-6ae2b55a2f37 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:57.743596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-eac4af22-c583-41cb-ad2f-6ae2b55a2f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:57.830185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:57.864614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 199/389] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:23:57.872611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac4af22-c583-41cb-ad2f-6ae2b55a2f37 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:57.915764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4503919-c188-402c-aaac-16007c13046f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:58.006264 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac4af22-c583-41cb-ad2f-6ae2b55a2f37 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:58.043782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963ce40-a4fa-4af3-9d99-d9eb564844e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:58.044810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963ce40-a4fa-4af3-9d99-d9eb564844e8 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:58.045323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963ce40-a4fa-4af3-9d99-d9eb564844e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:58.045806 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963ce40-a4fa-4af3-9d99-d9eb564844e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:58.057734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9963ce40-a4fa-4af3-9d99-d9eb564844e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-9963ce40-a4fa-4af3-9d99-d9eb564844e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:58.063777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:58.064719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:58.065512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:58.065512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:58.065512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:58.065512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:58.065512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:58.065512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:58.065512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:58.065512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:58.065512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:58.065512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:58.065512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:58.065512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:58.068472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 200/390] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:23:58.073889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce161d-853b-4a89-ab4e-b265722fbe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:58.073889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce161d-853b-4a89-ab4e-b265722fbe08 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:58.073889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce161d-853b-4a89-ab4e-b265722fbe08 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:58.073889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce161d-853b-4a89-ab4e-b265722fbe08 None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:58.074250 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce161d-853b-4a89-ab4e-b265722fbe08 None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:58.077016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963ce40-a4fa-4af3-9d99-d9eb564844e8 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:58.087006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce161d-853b-4a89-ab4e-b265722fbe08 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:58.096249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 191/391] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:57 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:58.102194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce161d-853b-4a89-ab4e-b265722fbe08 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:58.147370 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:58.207230 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963ce40-a4fa-4af3-9d99-d9eb564844e8 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:58.247541 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc87ca-c408-4bc5-bdbf-59a91cea9b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:58.248553 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc87ca-c408-4bc5-bdbf-59a91cea9b79 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:58.249086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc87ca-c408-4bc5-bdbf-59a91cea9b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:58.249603 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc87ca-c408-4bc5-bdbf-59a91cea9b79 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:58.252016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc87ca-c408-4bc5-bdbf-59a91cea9b79 None admin] RBAC: Adding query filter params (name=member) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:58.252550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc87ca-c408-4bc5-bdbf-59a91cea9b79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:58.254142 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc87ca-c408-4bc5-bdbf-59a91cea9b79 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:58.265544 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 201/392] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:23:58.274339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc87ca-c408-4bc5-bdbf-59a91cea9b79 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:58.341186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:58.406041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce161d-853b-4a89-ab4e-b265722fbe08 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:58.461176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05621993-b053-4c46-aae5-48378bb54ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:58.462016 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05621993-b053-4c46-aae5-48378bb54ba7 None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:58.462295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05621993-b053-4c46-aae5-48378bb54ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:58.463187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05621993-b053-4c46-aae5-48378bb54ba7 None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e/groups/b8324c97af7b4541b768ec0f4a9ae903/roles/93f699f03f0f41498415ab9533bdfcb4` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:58.465648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05621993-b053-4c46-aae5-48378bb54ba7 None admin] RBAC: Authorizing `identity:create_grant(project_id=703ed0f0d1ff4514b6dc0e857cbf989e, group_id=b8324c97af7b4541b768ec0f4a9ae903, role_id=93f699f03f0f41498415ab9533bdfcb4)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:58.468464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05621993-b053-4c46-aae5-48378bb54ba7 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:58.483364 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc87ca-c408-4bc5-bdbf-59a91cea9b79 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:58.493872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:58.524284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71da57c-5ac8-4272-b7b5-db2b2a13f9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:58.525879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71da57c-5ac8-4272-b7b5-db2b2a13f9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:58.526594 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71da57c-5ac8-4272-b7b5-db2b2a13f9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:58.526960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71da57c-5ac8-4272-b7b5-db2b2a13f9f9 None admin] PATH_INFO: `/v3/groups/b8324c97af7b4541b768ec0f4a9ae903` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:58.528830 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71da57c-5ac8-4272-b7b5-db2b2a13f9f9 None admin] RBAC: Authorizing `identity:get_group(group_id=b8324c97af7b4541b768ec0f4a9ae903)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:58.531185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71da57c-5ac8-4272-b7b5-db2b2a13f9f9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:58.539474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 202/393] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:23:58 2022] GET /identity/v3/groups/b8324c97af7b4541b768ec0f4a9ae903 => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:58.539474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71da57c-5ac8-4272-b7b5-db2b2a13f9f9 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:58.550225 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 192/394] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Sep 16 05:23:58 2022] PUT /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e/groups/b8324c97af7b4541b768ec0f4a9ae903/roles/93f699f03f0f41498415ab9533bdfcb4 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:23:58.551303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05621993-b053-4c46-aae5-48378bb54ba7 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:58.690103 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71da57c-5ac8-4272-b7b5-db2b2a13f9f9 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:58.723700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05621993-b053-4c46-aae5-48378bb54ba7 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:58.730241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f04b76-7ef2-4820-9b54-9e2d54afba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:58.731251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f04b76-7ef2-4820-9b54-9e2d54afba7c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:58.731696 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f04b76-7ef2-4820-9b54-9e2d54afba7c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:58.732142 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f04b76-7ef2-4820-9b54-9e2d54afba7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:58.746508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f04b76-7ef2-4820-9b54-9e2d54afba7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-51f04b76-7ef2-4820-9b54-9e2d54afba7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:58.752958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:58.753866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:58.754672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:58.754672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:58.754672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:58.754672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:58.754672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:58.754672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:58.754672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:58.754672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:58.754672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:58.754672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:58.754672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:23:58.754672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:58.754672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 203/395] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:23:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:58.756257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88480c03-2990-4a40-ac4a-30c53e0718cf None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:58.756257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88480c03-2990-4a40-ac4a-30c53e0718cf None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:58.756257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88480c03-2990-4a40-ac4a-30c53e0718cf None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:58.757888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 204/396] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:58.761088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88480c03-2990-4a40-ac4a-30c53e0718cf None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:58.778965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67440ab2-2c54-44f6-8b51-264b7639e73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:58.779600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67440ab2-2c54-44f6-8b51-264b7639e73e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:58.779877 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67440ab2-2c54-44f6-8b51-264b7639e73e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:58.780145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67440ab2-2c54-44f6-8b51-264b7639e73e None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:58.781413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67440ab2-2c54-44f6-8b51-264b7639e73e None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:58.782998 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67440ab2-2c54-44f6-8b51-264b7639e73e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:58.784499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 193/397] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:58 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:58.785603 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73991288-fb6e-46d0-87a0-939bc38633cb None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:58.786223 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73991288-fb6e-46d0-87a0-939bc38633cb None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:58.786223 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73991288-fb6e-46d0-87a0-939bc38633cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:58.792305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:58.809219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:58.811453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-73991288-fb6e-46d0-87a0-939bc38633cb None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:58.812522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-73991288-fb6e-46d0-87a0-939bc38633cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:58.886528 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88480c03-2990-4a40-ac4a-30c53e0718cf None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:58.892514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 194/398] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:58.893173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73991288-fb6e-46d0-87a0-939bc38633cb None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:58.942386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e999d0-dcdc-4a77-8fec-63d13f171921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:58.943084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e999d0-dcdc-4a77-8fec-63d13f171921 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:58.943376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e999d0-dcdc-4a77-8fec-63d13f171921 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:58.943641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e999d0-dcdc-4a77-8fec-63d13f171921 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:58.944362 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e999d0-dcdc-4a77-8fec-63d13f171921 None admin] RBAC: Adding query filter params (name=member) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:58.944653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e999d0-dcdc-4a77-8fec-63d13f171921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:58.946161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e999d0-dcdc-4a77-8fec-63d13f171921 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:58.962703 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 205/399] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:23:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:58.966820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e999d0-dcdc-4a77-8fec-63d13f171921 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:59.019776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73991288-fb6e-46d0-87a0-939bc38633cb None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:59.052890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:59.058770 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad945d2-cfae-49cc-accc-370c26b2248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:59.059548 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad945d2-cfae-49cc-accc-370c26b2248d None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:59.059819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad945d2-cfae-49cc-accc-370c26b2248d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:59.060084 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad945d2-cfae-49cc-accc-370c26b2248d None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e/groups/b8324c97af7b4541b768ec0f4a9ae903/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:59.062137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad945d2-cfae-49cc-accc-370c26b2248d None admin] RBAC: Authorizing `identity:create_grant(project_id=703ed0f0d1ff4514b6dc0e857cbf989e, group_id=b8324c97af7b4541b768ec0f4a9ae903, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:59.068113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad945d2-cfae-49cc-accc-370c26b2248d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:59.117034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:59.123872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e999d0-dcdc-4a77-8fec-63d13f171921 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:59.147056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 195/400] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Sep 16 05:23:58 2022] PUT /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e/groups/b8324c97af7b4541b768ec0f4a9ae903/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:23:59.180769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94cbe3-a47e-452e-93b2-2eb293455d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:59.181439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94cbe3-a47e-452e-93b2-2eb293455d35 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:59.181713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94cbe3-a47e-452e-93b2-2eb293455d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:59.182016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94cbe3-a47e-452e-93b2-2eb293455d35 None admin] PATH_INFO: `/v3/groups/b8324c97af7b4541b768ec0f4a9ae903` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:59.183320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94cbe3-a47e-452e-93b2-2eb293455d35 None admin] RBAC: Authorizing `identity:get_group(group_id=b8324c97af7b4541b768ec0f4a9ae903)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:59.184799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94cbe3-a47e-452e-93b2-2eb293455d35 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:59.186830 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 206/401] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:23:58 2022] GET /identity/v3/groups/b8324c97af7b4541b768ec0f4a9ae903 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:59.195047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad945d2-cfae-49cc-accc-370c26b2248d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:59.206914 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94cbe3-a47e-452e-93b2-2eb293455d35 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:59.357087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad945d2-cfae-49cc-accc-370c26b2248d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:59.374437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94cbe3-a47e-452e-93b2-2eb293455d35 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:59.411240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcb259-097d-42fd-874f-4a444138975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:59.411874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcb259-097d-42fd-874f-4a444138975b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:59.412144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcb259-097d-42fd-874f-4a444138975b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:59.412403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcb259-097d-42fd-874f-4a444138975b None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:59.414371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcb259-097d-42fd-874f-4a444138975b None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:59.415809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcb259-097d-42fd-874f-4a444138975b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:59.417591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 196/402] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:59 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:59.419084 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a72f9-e7e9-49dc-b5ae-a8ace60eb28b None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:59.419698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a72f9-e7e9-49dc-b5ae-a8ace60eb28b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:59.420259 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a72f9-e7e9-49dc-b5ae-a8ace60eb28b None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:59.420662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7c1bd-4b95-4a96-94aa-77fc716f5d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:59.421291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7c1bd-4b95-4a96-94aa-77fc716f5d67 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:59.421566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7c1bd-4b95-4a96-94aa-77fc716f5d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:59.422270 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7c1bd-4b95-4a96-94aa-77fc716f5d67 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:59.422637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 197/403] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:23:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:23:59.423700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a72f9-e7e9-49dc-b5ae-a8ace60eb28b None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:59.441105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d7c1bd-4b95-4a96-94aa-77fc716f5d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-d5d7c1bd-4b95-4a96-94aa-77fc716f5d67 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:23:59.448621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:23:59.449534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:23:59.450394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:23:59.450394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:23:59.450394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:23:59.450394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:23:59.450394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:23:59.450394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:23:59.450394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:23:59.450394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:23:59.450394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:23:59.450394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:23:59.450394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:23:59.450394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:23:59.452151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 207/404] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:23:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:23:59.453781 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9ee31-502c-4080-a048-3f1aca753ff4 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:59.454278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9ee31-502c-4080-a048-3f1aca753ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:59.454747 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a9ee31-502c-4080-a048-3f1aca753ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:59.477675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a8a9ee31-502c-4080-a048-3f1aca753ff4 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:23:59.479856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a8a9ee31-502c-4080-a048-3f1aca753ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:23:59.500321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:59.506898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:59.544839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a72f9-e7e9-49dc-b5ae-a8ace60eb28b None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:59.574023 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 208/405] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:23:59.578959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9ee31-502c-4080-a048-3f1aca753ff4 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:59.602683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63abb5-710b-491b-8b64-85128dcd6f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:59.603765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63abb5-710b-491b-8b64-85128dcd6f7c None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:59.604039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63abb5-710b-491b-8b64-85128dcd6f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:59.604309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63abb5-710b-491b-8b64-85128dcd6f7c None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe/groups/b8324c97af7b4541b768ec0f4a9ae903/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:59.609406 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63abb5-710b-491b-8b64-85128dcd6f7c None admin] RBAC: Authorizing `identity:create_grant(project_id=082a3596b7724c4f8f59b2687c2ca0fe, group_id=b8324c97af7b4541b768ec0f4a9ae903, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:59.610855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63abb5-710b-491b-8b64-85128dcd6f7c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:59.708162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 198/406] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Sep 16 05:23:59 2022] PUT /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe/groups/b8324c97af7b4541b768ec0f4a9ae903/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:23:59.719755 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a9ee31-502c-4080-a048-3f1aca753ff4 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:59.780801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10717f4b-52b5-421a-a2ae-6f6a7cfb2c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:59.782145 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10717f4b-52b5-421a-a2ae-6f6a7cfb2c02 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:59.782541 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10717f4b-52b5-421a-a2ae-6f6a7cfb2c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:59.782816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10717f4b-52b5-421a-a2ae-6f6a7cfb2c02 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:59.783548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10717f4b-52b5-421a-a2ae-6f6a7cfb2c02 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:23:59.783845 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10717f4b-52b5-421a-a2ae-6f6a7cfb2c02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:59.785274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10717f4b-52b5-421a-a2ae-6f6a7cfb2c02 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:59.799590 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 209/407] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:23:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:59.801174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63abb5-710b-491b-8b64-85128dcd6f7c None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:59.925849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63abb5-710b-491b-8b64-85128dcd6f7c None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:23:59.932644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10717f4b-52b5-421a-a2ae-6f6a7cfb2c02 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:23:59.948348 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:59.956784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:23:59.962635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d3677b-45ae-439e-a102-d90b6c41027b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:23:59.963263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3677b-45ae-439e-a102-d90b6c41027b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:23:59.963536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3677b-45ae-439e-a102-d90b6c41027b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:23:59.963803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3677b-45ae-439e-a102-d90b6c41027b None admin] PATH_INFO: `/v3/groups/b8324c97af7b4541b768ec0f4a9ae903` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:23:59.964905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d3677b-45ae-439e-a102-d90b6c41027b None admin] RBAC: Authorizing `identity:get_group(group_id=b8324c97af7b4541b768ec0f4a9ae903)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:23:59.966367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d3677b-45ae-439e-a102-d90b6c41027b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:23:59.968017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 199/408] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:23:59 2022] GET /identity/v3/groups/b8324c97af7b4541b768ec0f4a9ae903 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:23:59.974645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d3677b-45ae-439e-a102-d90b6c41027b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:00.069549 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10717f4b-52b5-421a-a2ae-6f6a7cfb2c02 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:00.083699 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d3677b-45ae-439e-a102-d90b6c41027b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:00.124580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f6e70-e3e1-4a90-a44f-22342efd2f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:00.125307 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f6e70-e3e1-4a90-a44f-22342efd2f16 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:00.126303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f6e70-e3e1-4a90-a44f-22342efd2f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:00.126578 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f6e70-e3e1-4a90-a44f-22342efd2f16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:00.128540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefed0c-ebde-49fe-ac20-6c83940fc059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:00.129162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefed0c-ebde-49fe-ac20-6c83940fc059 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:00.129432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefed0c-ebde-49fe-ac20-6c83940fc059 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:00.129697 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefed0c-ebde-49fe-ac20-6c83940fc059 None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:00.130923 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefed0c-ebde-49fe-ac20-6c83940fc059 None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:00.132342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefed0c-ebde-49fe-ac20-6c83940fc059 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:00.134217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 200/409] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:23:59 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:00.140431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefed0c-ebde-49fe-ac20-6c83940fc059 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:00.155666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f6e70-e3e1-4a90-a44f-22342efd2f16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-511f6e70-e3e1-4a90-a44f-22342efd2f16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:00.163271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:00.164153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:00.165196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:00.165196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:00.165196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:00.165196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:00.165196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:00.165196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:00.165196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:00.165196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:00.165196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:00.165196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:00.165196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:00.165196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:00.165196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 210/410] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:23:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:00.174928 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f6e70-e3e1-4a90-a44f-22342efd2f16 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:00.281178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:00.283339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:00.328772 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f6e70-e3e1-4a90-a44f-22342efd2f16 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:00.369674 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefed0c-ebde-49fe-ac20-6c83940fc059 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:00.389164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622c241-7cae-4c2a-b96f-0b0688cb0291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:00.389859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622c241-7cae-4c2a-b96f-0b0688cb0291 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:00.390214 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622c241-7cae-4c2a-b96f-0b0688cb0291 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:00.390480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622c241-7cae-4c2a-b96f-0b0688cb0291 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:00.392055 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622c241-7cae-4c2a-b96f-0b0688cb0291 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:00.392354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622c241-7cae-4c2a-b96f-0b0688cb0291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:00.393731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622c241-7cae-4c2a-b96f-0b0688cb0291 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:00.406235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc737e-5a2a-427a-b6d0-f29f16ce9c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:00.407798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc737e-5a2a-427a-b6d0-f29f16ce9c61 None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:00.408254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc737e-5a2a-427a-b6d0-f29f16ce9c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:00.408697 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc737e-5a2a-427a-b6d0-f29f16ce9c61 None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe/groups/b8324c97af7b4541b768ec0f4a9ae903/roles/93f699f03f0f41498415ab9533bdfcb4` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:00.411055 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc737e-5a2a-427a-b6d0-f29f16ce9c61 None admin] RBAC: Authorizing `identity:create_grant(project_id=082a3596b7724c4f8f59b2687c2ca0fe, group_id=b8324c97af7b4541b768ec0f4a9ae903, role_id=93f699f03f0f41498415ab9533bdfcb4)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:00.411605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 211/411] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:00.412918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc737e-5a2a-427a-b6d0-f29f16ce9c61 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:00.416958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622c241-7cae-4c2a-b96f-0b0688cb0291 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:00.485695 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 201/412] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Sep 16 05:24:00 2022] PUT /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe/groups/b8324c97af7b4541b768ec0f4a9ae903/roles/93f699f03f0f41498415ab9533bdfcb4 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:24:00.539247 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622c241-7cae-4c2a-b96f-0b0688cb0291 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:00.587642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff65e3-7665-46d0-9be3-cfc4afd45ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:00.588677 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff65e3-7665-46d0-9be3-cfc4afd45ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:00.589133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff65e3-7665-46d0-9be3-cfc4afd45ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:00.589609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff65e3-7665-46d0-9be3-cfc4afd45ea5 None admin] PATH_INFO: `/v3/groups/d6bee4d7fb124734a8168a39e29d938d` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:00.595421 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff65e3-7665-46d0-9be3-cfc4afd45ea5 None admin] RBAC: Authorizing `identity:get_group(group_id=d6bee4d7fb124734a8168a39e29d938d)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:00.601705 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff65e3-7665-46d0-9be3-cfc4afd45ea5 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:00.601705 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 212/413] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:24:00 2022] GET /identity/v3/groups/d6bee4d7fb124734a8168a39e29d938d => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:00.605006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc737e-5a2a-427a-b6d0-f29f16ce9c61 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:00.730555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc737e-5a2a-427a-b6d0-f29f16ce9c61 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:00.755231 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:00.777419 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cffcbe-db52-465a-99fe-55113153f84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:00.778079 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cffcbe-db52-465a-99fe-55113153f84c None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:00.778355 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cffcbe-db52-465a-99fe-55113153f84c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:00.778616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cffcbe-db52-465a-99fe-55113153f84c None admin] PATH_INFO: `/v3/projects/2c317c5acb304c5d983c73b9ed1b6802` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:00.779745 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cffcbe-db52-465a-99fe-55113153f84c None admin] RBAC: Authorizing `identity:get_project(project_id=2c317c5acb304c5d983c73b9ed1b6802)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:00.781167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cffcbe-db52-465a-99fe-55113153f84c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:00.784401 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 202/414] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:24:00 2022] GET /identity/v3/projects/2c317c5acb304c5d983c73b9ed1b6802 => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:00.789931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff65e3-7665-46d0-9be3-cfc4afd45ea5 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:00.800315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:00.953803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff65e3-7665-46d0-9be3-cfc4afd45ea5 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:01.000408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a56dd3-673f-40ac-a4ed-031df5c4cbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:01.001139 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a56dd3-673f-40ac-a4ed-031df5c4cbf2 None admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:01.001410 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a56dd3-673f-40ac-a4ed-031df5c4cbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:01.001676 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a56dd3-673f-40ac-a4ed-031df5c4cbf2 None admin] PATH_INFO: `/v3/projects/2c317c5acb304c5d983c73b9ed1b6802/groups/d6bee4d7fb124734a8168a39e29d938d/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:01.003581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a56dd3-673f-40ac-a4ed-031df5c4cbf2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c317c5acb304c5d983c73b9ed1b6802, group_id=d6bee4d7fb124734a8168a39e29d938d, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:01.005009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a56dd3-673f-40ac-a4ed-031df5c4cbf2 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:01.085731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 213/415] 10.222.0.49 () {58 vars in 1442 bytes} [Fri Sep 16 05:24:00 2022] PUT /identity/v3/projects/2c317c5acb304c5d983c73b9ed1b6802/groups/d6bee4d7fb124734a8168a39e29d938d/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:24:01.240192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b4f03-ef44-4c8c-845e-c0aacc40fb71 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:01.240843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b4f03-ef44-4c8c-845e-c0aacc40fb71 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:01.241418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b4f03-ef44-4c8c-845e-c0aacc40fb71 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:01.243037 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 203/416] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:01.249090 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027ced0-d42c-414f-9df8-8d5d05fe4c0f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:01.249701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027ced0-d42c-414f-9df8-8d5d05fe4c0f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:01.250296 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027ced0-d42c-414f-9df8-8d5d05fe4c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:01.278478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-b027ced0-d42c-414f-9df8-8d5d05fe4c0f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:01.280089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b027ced0-d42c-414f-9df8-8d5d05fe4c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:01.351811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:01.359336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 214/417] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:01.727969 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b4f03-ef44-4c8c-845e-c0aacc40fb71 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:01.851112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b4f03-ef44-4c8c-845e-c0aacc40fb71 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:01.890570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ac634-8ece-4493-9046-efa8148f888f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:01.892309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ac634-8ece-4493-9046-efa8148f888f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:01.892309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ac634-8ece-4493-9046-efa8148f888f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:01.892615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ac634-8ece-4493-9046-efa8148f888f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:01.907496 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ac634-8ece-4493-9046-efa8148f888f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-e96ac634-8ece-4493-9046-efa8148f888f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:01.912244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:01.913112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:01.913951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:01.913951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:01.913951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:01.913951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:01.913951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:01.913951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:01.913951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:01.913951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:01.913951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:01.913951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:01.913951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:24:01.913951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:01.913951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 204/418] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:24:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:01.923482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027ced0-d42c-414f-9df8-8d5d05fe4c0f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:02.046280 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cee97f-c2f4-424e-ba4f-8b12bab8775e None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:02.046465 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cee97f-c2f4-424e-ba4f-8b12bab8775e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:02.046731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cee97f-c2f4-424e-ba4f-8b12bab8775e None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:02.049306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 205/419] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:24:02.061616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b20d8-6095-41fc-922e-48ac6d71af6d None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:02.061944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b20d8-6095-41fc-922e-48ac6d71af6d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:02.062216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b20d8-6095-41fc-922e-48ac6d71af6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:02.080390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027ced0-d42c-414f-9df8-8d5d05fe4c0f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:02.100638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f05b20d8-6095-41fc-922e-48ac6d71af6d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:02.102878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f05b20d8-6095-41fc-922e-48ac6d71af6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:02.139086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1e97c-6741-4db0-afa9-05791a647809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:02.140042 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1e97c-6741-4db0-afa9-05791a647809 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:02.140497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1e97c-6741-4db0-afa9-05791a647809 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:02.141777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1e97c-6741-4db0-afa9-05791a647809 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:02.142742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1e97c-6741-4db0-afa9-05791a647809 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:02.143240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1e97c-6741-4db0-afa9-05791a647809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:02.144797 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1e97c-6741-4db0-afa9-05791a647809 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:02.153664 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 215/420] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:02.159786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1e97c-6741-4db0-afa9-05791a647809 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:02.202775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:02.228783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:02.270604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 206/421] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:02.305656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1e97c-6741-4db0-afa9-05791a647809 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:02.366725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4e52e-03fb-47a0-99d8-1cd8f1bbed9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:02.367334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4e52e-03fb-47a0-99d8-1cd8f1bbed9c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:02.367607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4e52e-03fb-47a0-99d8-1cd8f1bbed9c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:02.367871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4e52e-03fb-47a0-99d8-1cd8f1bbed9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:02.454887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4e52e-03fb-47a0-99d8-1cd8f1bbed9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:02.456699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4e52e-03fb-47a0-99d8-1cd8f1bbed9c None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:02.465267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe55c03-9c5f-4c6d-855b-01d1d6205277 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:02.465547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe55c03-9c5f-4c6d-855b-01d1d6205277 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:02.465815 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe55c03-9c5f-4c6d-855b-01d1d6205277 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:02.467157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 207/422] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:24:02.473457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a994ffa-d479-429f-9911-0d04773aede8 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:02.473729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a994ffa-d479-429f-9911-0d04773aede8 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:02.474027 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a994ffa-d479-429f-9911-0d04773aede8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:02.494219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-9a994ffa-d479-429f-9911-0d04773aede8 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:02.495624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9a994ffa-d479-429f-9911-0d04773aede8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:02.554428 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:02.586086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 208/423] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:24:02.599481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a994ffa-d479-429f-9911-0d04773aede8 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:02.639856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 216/424] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 05:24:02.648217 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4e52e-03fb-47a0-99d8-1cd8f1bbed9c None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:02.769998 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a994ffa-d479-429f-9911-0d04773aede8 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:02.774126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:02.818325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4e52e-03fb-47a0-99d8-1cd8f1bbed9c None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:02.832441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6491b-ad52-4943-9eb8-a07dbe9be821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:02.833073 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6491b-ad52-4943-9eb8-a07dbe9be821 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:02.833347 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6491b-ad52-4943-9eb8-a07dbe9be821 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:02.833611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6491b-ad52-4943-9eb8-a07dbe9be821 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:02.851741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6491b-ad52-4943-9eb8-a07dbe9be821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-1ef6491b-ad52-4943-9eb8-a07dbe9be821 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:02.856471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:02.857372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:02.858216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:02.858216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:02.858216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:02.858216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:02.858216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:02.858216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:02.858216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:02.858216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:02.858216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:02.858216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:02.858216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:24:02.858216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:02.858216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 209/425] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:24:02.868386 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6491b-ad52-4943-9eb8-a07dbe9be821 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:02.888819 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9aace-3ab8-4370-833d-76a777a1b482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:02.889455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9aace-3ab8-4370-833d-76a777a1b482 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:02.889723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9aace-3ab8-4370-833d-76a777a1b482 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:02.890019 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa9aace-3ab8-4370-833d-76a777a1b482 None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:02.891339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9aace-3ab8-4370-833d-76a777a1b482 None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:02.894271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa9aace-3ab8-4370-833d-76a777a1b482 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:02.897062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 217/426] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:24:02 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:02.902532 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9aace-3ab8-4370-833d-76a777a1b482 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:02.935111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:02.995784 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6491b-ad52-4943-9eb8-a07dbe9be821 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:03.010641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa9aace-3ab8-4370-833d-76a777a1b482 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:03.030273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7a635-7f43-428b-90df-a838777cd31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:03.031758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7a635-7f43-428b-90df-a838777cd31a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:03.032243 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7a635-7f43-428b-90df-a838777cd31a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:03.032714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7a635-7f43-428b-90df-a838777cd31a None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:03.033570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7a635-7f43-428b-90df-a838777cd31a None admin] RBAC: Adding query filter params (name=member) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:03.034108 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7a635-7f43-428b-90df-a838777cd31a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:03.035688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7a635-7f43-428b-90df-a838777cd31a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:03.049393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0837a0c-45cb-43f7-905c-ff41ee760870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:03.049393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0837a0c-45cb-43f7-905c-ff41ee760870 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:03.049393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0837a0c-45cb-43f7-905c-ff41ee760870 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:03.049393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0837a0c-45cb-43f7-905c-ff41ee760870 None admin] PATH_INFO: `/v3/groups/b8324c97af7b4541b768ec0f4a9ae903` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:03.049780 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0837a0c-45cb-43f7-905c-ff41ee760870 None admin] RBAC: Authorizing `identity:get_group(group_id=b8324c97af7b4541b768ec0f4a9ae903)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:03.050062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 210/427] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:24:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:03.050892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0837a0c-45cb-43f7-905c-ff41ee760870 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:03.051314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7a635-7f43-428b-90df-a838777cd31a None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:03.052876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 218/428] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:24:02 2022] GET /identity/v3/groups/b8324c97af7b4541b768ec0f4a9ae903 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:03.053690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0837a0c-45cb-43f7-905c-ff41ee760870 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:03.086278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:03.152472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:03.159601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0837a0c-45cb-43f7-905c-ff41ee760870 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:03.180895 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7a635-7f43-428b-90df-a838777cd31a None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:03.196696 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cc4e8-0d9f-4351-a12c-e3e2c607a341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:03.197404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cc4e8-0d9f-4351-a12c-e3e2c607a341 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:03.197769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cc4e8-0d9f-4351-a12c-e3e2c607a341 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:03.198107 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cc4e8-0d9f-4351-a12c-e3e2c607a341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:03.229531 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de601339-ff67-484c-9fbe-7bf6a2befd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:03.230569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de601339-ff67-484c-9fbe-7bf6a2befd13 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:03.231180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de601339-ff67-484c-9fbe-7bf6a2befd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:03.231642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de601339-ff67-484c-9fbe-7bf6a2befd13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:03.249460 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de601339-ff67-484c-9fbe-7bf6a2befd13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-de601339-ff67-484c-9fbe-7bf6a2befd13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:03.254859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:03.255846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:03.256630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:03.256630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:03.256630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:03.256630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:03.256630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:03.256630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:03.256630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:03.256630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:03.256630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:03.256630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:03.256630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 16 05:24:03.256630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:03.260853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 211/429] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:03.261856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de601339-ff67-484c-9fbe-7bf6a2befd13 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:03.288490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433cc4e8-0d9f-4351-a12c-e3e2c607a341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:03.291031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433cc4e8-0d9f-4351-a12c-e3e2c607a341 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:03.354331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:03.368738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de601339-ff67-484c-9fbe-7bf6a2befd13 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:03.398195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:03.404573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad12eba4-cf81-4feb-aa03-74580f1bde55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:03.405110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12eba4-cf81-4feb-aa03-74580f1bde55 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:03.405379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12eba4-cf81-4feb-aa03-74580f1bde55 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:03.405639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad12eba4-cf81-4feb-aa03-74580f1bde55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:03.406178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 219/430] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:24:03.406930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12eba4-cf81-4feb-aa03-74580f1bde55 None admin] RBAC: Adding query filter params (group.id=b8324c97af7b4541b768ec0f4a9ae903, role.id=93f699f03f0f41498415ab9533bdfcb4, scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:03.407230 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12eba4-cf81-4feb-aa03-74580f1bde55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:03.407761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cc4e8-0d9f-4351-a12c-e3e2c607a341 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:03.408703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad12eba4-cf81-4feb-aa03-74580f1bde55 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:03.418314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 212/431] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 16 05:24:03 2022] GET /identity/v3/role_assignments?group.id=b8324c97af7b4541b768ec0f4a9ae903&scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e&role.id=93f699f03f0f41498415ab9533bdfcb4 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:03.419798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272e450-0842-4393-b9d9-88b740d470dd None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:03.420256 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272e450-0842-4393-b9d9-88b740d470dd None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:03.420503 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272e450-0842-4393-b9d9-88b740d470dd None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:03.421967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 213/432] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:03.422741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272e450-0842-4393-b9d9-88b740d470dd None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:03.539123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cc4e8-0d9f-4351-a12c-e3e2c607a341 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:03.553528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272e450-0842-4393-b9d9-88b740d470dd None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:03.580405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6a9f6-b879-4454-adc1-5703803c4a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:03.581519 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6a9f6-b879-4454-adc1-5703803c4a14 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:03.582026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6a9f6-b879-4454-adc1-5703803c4a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:03.582572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6a9f6-b879-4454-adc1-5703803c4a14 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:03.583518 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6a9f6-b879-4454-adc1-5703803c4a14 None admin] RBAC: Adding query filter params (name=member) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:03.584018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6a9f6-b879-4454-adc1-5703803c4a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:03.585912 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6a9f6-b879-4454-adc1-5703803c4a14 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:03.595233 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175db803-40ba-4607-995a-2d42351b7135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:03.595878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175db803-40ba-4607-995a-2d42351b7135 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:03.596151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175db803-40ba-4607-995a-2d42351b7135 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:03.596419 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175db803-40ba-4607-995a-2d42351b7135 None admin] PATH_INFO: `/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:03.600022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 220/433] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:24:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:03.600129 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175db803-40ba-4607-995a-2d42351b7135 None admin] RBAC: Authorizing `identity:get_project(project_id=703ed0f0d1ff4514b6dc0e857cbf989e)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:03.600202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc679d1a-9c6f-48fa-bf20-bebfa9b5f8ba None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:03.600202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc679d1a-9c6f-48fa-bf20-bebfa9b5f8ba None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:03.600202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc679d1a-9c6f-48fa-bf20-bebfa9b5f8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:03.600372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175db803-40ba-4607-995a-2d42351b7135 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:03.601317 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 214/434] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:24:03 2022] GET /identity/v3/projects/703ed0f0d1ff4514b6dc0e857cbf989e => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:03.604205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175db803-40ba-4607-995a-2d42351b7135 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:03.630619 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-fc679d1a-9c6f-48fa-bf20-bebfa9b5f8ba None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:03.632305 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-fc679d1a-9c6f-48fa-bf20-bebfa9b5f8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:03.632833 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:03.697202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:03.714982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 221/435] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:03.719846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc679d1a-9c6f-48fa-bf20-bebfa9b5f8ba None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:03.739137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175db803-40ba-4607-995a-2d42351b7135 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:03.779627 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea403466-3e69-499f-9935-96693d46eec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:03.780272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea403466-3e69-499f-9935-96693d46eec6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:03.780590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea403466-3e69-499f-9935-96693d46eec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:03.780851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea403466-3e69-499f-9935-96693d46eec6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:03.820951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc679d1a-9c6f-48fa-bf20-bebfa9b5f8ba None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:03.854302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea403466-3e69-499f-9935-96693d46eec6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:03.856142 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c08be7-64c5-4ab1-a6fb-03e0b55d7d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:03.856419 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea403466-3e69-499f-9935-96693d46eec6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:03.856773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c08be7-64c5-4ab1-a6fb-03e0b55d7d13 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:03.857049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c08be7-64c5-4ab1-a6fb-03e0b55d7d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:03.857316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c08be7-64c5-4ab1-a6fb-03e0b55d7d13 None admin] PATH_INFO: `/v3/groups/b8324c97af7b4541b768ec0f4a9ae903` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:03.858435 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c08be7-64c5-4ab1-a6fb-03e0b55d7d13 None admin] RBAC: Authorizing `identity:get_group(group_id=b8324c97af7b4541b768ec0f4a9ae903)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:03.859900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c08be7-64c5-4ab1-a6fb-03e0b55d7d13 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:03.861454 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 222/436] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:24:03 2022] GET /identity/v3/groups/b8324c97af7b4541b768ec0f4a9ae903 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:03.862596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a42fd-2622-47dd-883c-38a339c0836c None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:03.862874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a42fd-2622-47dd-883c-38a339c0836c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:03.863143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a42fd-2622-47dd-883c-38a339c0836c None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:03.864323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 223/437] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:03.867076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a42fd-2622-47dd-883c-38a339c0836c None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:03.879054 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:03.931395 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:03.971771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 215/438] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 05:24:03.973217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c3066-6f85-4df8-be70-843906fa5622 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:03.973503 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c3066-6f85-4df8-be70-843906fa5622 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:03.973774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c3066-6f85-4df8-be70-843906fa5622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:03.985131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a42fd-2622-47dd-883c-38a339c0836c None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:03.993470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-7a8c3066-6f85-4df8-be70-843906fa5622 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:03.994809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7a8c3066-6f85-4df8-be70-843906fa5622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:04.019427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f7931-2bf5-4cc6-95fa-ee608f7bccb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:04.020123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f7931-2bf5-4cc6-95fa-ee608f7bccb6 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:04.020400 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f7931-2bf5-4cc6-95fa-ee608f7bccb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:04.020668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f7931-2bf5-4cc6-95fa-ee608f7bccb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:04.021385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f7931-2bf5-4cc6-95fa-ee608f7bccb6 None admin] RBAC: Adding query filter params (group.id=b8324c97af7b4541b768ec0f4a9ae903, role.id=413cc5f8baab440bab827a3547b65f33, scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:04.021686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f7931-2bf5-4cc6-95fa-ee608f7bccb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:04.024251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f7931-2bf5-4cc6-95fa-ee608f7bccb6 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:04.034032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 224/439] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 16 05:24:03 2022] GET /identity/v3/role_assignments?group.id=b8324c97af7b4541b768ec0f4a9ae903&scope.project.id=703ed0f0d1ff4514b6dc0e857cbf989e&role.id=413cc5f8baab440bab827a3547b65f33 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:04.034032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f7931-2bf5-4cc6-95fa-ee608f7bccb6 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:04.070088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 216/440] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:04.071254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c3066-6f85-4df8-be70-843906fa5622 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:04.130732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:04.172076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f7931-2bf5-4cc6-95fa-ee608f7bccb6 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:04.198862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:04.204110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c3066-6f85-4df8-be70-843906fa5622 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:04.225100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e94b2-fcb3-4a15-b059-47d327ed034e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:04.225922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e94b2-fcb3-4a15-b059-47d327ed034e None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:04.226212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e94b2-fcb3-4a15-b059-47d327ed034e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:04.226486 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e94b2-fcb3-4a15-b059-47d327ed034e None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:04.227957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e94b2-fcb3-4a15-b059-47d327ed034e None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:04.229920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e94b2-fcb3-4a15-b059-47d327ed034e None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:04.231623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 225/441] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:24:04 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:04.238599 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e94b2-fcb3-4a15-b059-47d327ed034e None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:04.261116 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b8cad-bda8-4fc4-96fe-e41e187df982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:04.261885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b8cad-bda8-4fc4-96fe-e41e187df982 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:04.262170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b8cad-bda8-4fc4-96fe-e41e187df982 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:04.262438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b8cad-bda8-4fc4-96fe-e41e187df982 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:04.275836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b8cad-bda8-4fc4-96fe-e41e187df982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-918b8cad-bda8-4fc4-96fe-e41e187df982 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:04.281269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:04.282652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:04.283436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:04.283436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:04.283436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:04.283436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:04.283436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:04.283436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:04.283436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:04.283436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:04.283436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:04.283436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:04.283436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:24:04.283436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:04.284705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 217/442] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:24:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:04.291467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b8cad-bda8-4fc4-96fe-e41e187df982 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:04.368604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e94b2-fcb3-4a15-b059-47d327ed034e None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:04.414161 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b8cad-bda8-4fc4-96fe-e41e187df982 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:04.414510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f56032-1501-4dfb-9ad7-ccbc3dea498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:04.415156 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f56032-1501-4dfb-9ad7-ccbc3dea498f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:04.415430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f56032-1501-4dfb-9ad7-ccbc3dea498f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:04.415873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f56032-1501-4dfb-9ad7-ccbc3dea498f None admin] PATH_INFO: `/v3/groups/b8324c97af7b4541b768ec0f4a9ae903` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:04.417026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f56032-1501-4dfb-9ad7-ccbc3dea498f None admin] RBAC: Authorizing `identity:get_group(group_id=b8324c97af7b4541b768ec0f4a9ae903)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:04.418487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f56032-1501-4dfb-9ad7-ccbc3dea498f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:04.425328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:04.425644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 226/443] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:24:04 2022] GET /identity/v3/groups/b8324c97af7b4541b768ec0f4a9ae903 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:04.426586 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f56032-1501-4dfb-9ad7-ccbc3dea498f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:04.455714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9ec40-868a-462b-b614-c4c22dd42809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:04.456571 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9ec40-868a-462b-b614-c4c22dd42809 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:04.457104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9ec40-868a-462b-b614-c4c22dd42809 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:04.457369 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9ec40-868a-462b-b614-c4c22dd42809 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:04.458055 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9ec40-868a-462b-b614-c4c22dd42809 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:04.458349 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9ec40-868a-462b-b614-c4c22dd42809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:04.459859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd9ec40-868a-462b-b614-c4c22dd42809 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:04.470769 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 218/444] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:04.471585 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9ec40-868a-462b-b614-c4c22dd42809 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:04.496104 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:04.550216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f56032-1501-4dfb-9ad7-ccbc3dea498f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:04.585453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9ec40-868a-462b-b614-c4c22dd42809 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:04.596047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3869f6-2f90-4001-a8e5-ac211ebe9c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:04.596679 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3869f6-2f90-4001-a8e5-ac211ebe9c41 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:04.596949 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3869f6-2f90-4001-a8e5-ac211ebe9c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:04.597212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3869f6-2f90-4001-a8e5-ac211ebe9c41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:04.607998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3869f6-2f90-4001-a8e5-ac211ebe9c41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-db3869f6-2f90-4001-a8e5-ac211ebe9c41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:04.614148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:04.615219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 227/445] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:24:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:04.616033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3869f6-2f90-4001-a8e5-ac211ebe9c41 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:04.635247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe51ae-78f1-4c6e-b6b8-ce586fb2ee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:04.635901 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe51ae-78f1-4c6e-b6b8-ce586fb2ee38 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:04.636173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe51ae-78f1-4c6e-b6b8-ce586fb2ee38 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:04.636439 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe51ae-78f1-4c6e-b6b8-ce586fb2ee38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:04.637156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe51ae-78f1-4c6e-b6b8-ce586fb2ee38 None admin] RBAC: Adding query filter params (group.id=b8324c97af7b4541b768ec0f4a9ae903, role.id=413cc5f8baab440bab827a3547b65f33, scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:04.637456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe51ae-78f1-4c6e-b6b8-ce586fb2ee38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:04.639123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe51ae-78f1-4c6e-b6b8-ce586fb2ee38 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:04.645430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 219/446] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 16 05:24:04 2022] GET /identity/v3/role_assignments?group.id=b8324c97af7b4541b768ec0f4a9ae903&scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe&role.id=413cc5f8baab440bab827a3547b65f33 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:04.646302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe51ae-78f1-4c6e-b6b8-ce586fb2ee38 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:04.694235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:04.739207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3869f6-2f90-4001-a8e5-ac211ebe9c41 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:04.755120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:04.776422 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1394354-d098-40b9-a55b-ec7cd689fd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:04.776791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe51ae-78f1-4c6e-b6b8-ce586fb2ee38 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:04.777052 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1394354-d098-40b9-a55b-ec7cd689fd5a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:04.777328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1394354-d098-40b9-a55b-ec7cd689fd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:04.777599 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1394354-d098-40b9-a55b-ec7cd689fd5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:04.815657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e0677-4687-4919-8edf-659209a8e408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:04.816250 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e0677-4687-4919-8edf-659209a8e408 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:04.816515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e0677-4687-4919-8edf-659209a8e408 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:04.816780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e0677-4687-4919-8edf-659209a8e408 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:04.817440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e0677-4687-4919-8edf-659209a8e408 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:04.817731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e0677-4687-4919-8edf-659209a8e408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:04.819121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e0677-4687-4919-8edf-659209a8e408 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:04.829293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 220/447] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:04.835283 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e0677-4687-4919-8edf-659209a8e408 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:04.855227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1394354-d098-40b9-a55b-ec7cd689fd5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:04.856680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1394354-d098-40b9-a55b-ec7cd689fd5a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:04.950065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:04.959034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e0677-4687-4919-8edf-659209a8e408 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:04.982899 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 228/448] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:24:04.988446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1394354-d098-40b9-a55b-ec7cd689fd5a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:05.000617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a32fa-cca8-4608-9ef3-cc52653130a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:05.001235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a32fa-cca8-4608-9ef3-cc52653130a0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:05.001511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a32fa-cca8-4608-9ef3-cc52653130a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:05.001776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a32fa-cca8-4608-9ef3-cc52653130a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:05.006501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:05.077290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a32fa-cca8-4608-9ef3-cc52653130a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:05.078897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a32fa-cca8-4608-9ef3-cc52653130a0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:05.093194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1394354-d098-40b9-a55b-ec7cd689fd5a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:05.127431 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d771b-9e6a-4b2f-a733-60c49b8726b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:05.128050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d771b-9e6a-4b2f-a733-60c49b8726b1 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:05.128329 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d771b-9e6a-4b2f-a733-60c49b8726b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:05.128598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d771b-9e6a-4b2f-a733-60c49b8726b1 None admin] PATH_INFO: `/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:05.129706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d771b-9e6a-4b2f-a733-60c49b8726b1 None admin] RBAC: Authorizing `identity:get_project(project_id=082a3596b7724c4f8f59b2687c2ca0fe)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:05.131195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d771b-9e6a-4b2f-a733-60c49b8726b1 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:05.132801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 229/449] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:24:04 2022] GET /identity/v3/projects/082a3596b7724c4f8f59b2687c2ca0fe => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:05.138176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d771b-9e6a-4b2f-a733-60c49b8726b1 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:05.169381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:05.176554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 221/450] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 05:24:05.181909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a32fa-cca8-4608-9ef3-cc52653130a0 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:05.218027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:05.235826 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d771b-9e6a-4b2f-a733-60c49b8726b1 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:05.271744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2503483-cd7d-4392-93b5-4b8a28ed4996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:05.272293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2503483-cd7d-4392-93b5-4b8a28ed4996 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:05.272562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2503483-cd7d-4392-93b5-4b8a28ed4996 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:05.272827 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2503483-cd7d-4392-93b5-4b8a28ed4996 None admin] PATH_INFO: `/v3/groups/b8324c97af7b4541b768ec0f4a9ae903` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:05.273954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2503483-cd7d-4392-93b5-4b8a28ed4996 None admin] RBAC: Authorizing `identity:get_group(group_id=b8324c97af7b4541b768ec0f4a9ae903)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:05.275386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2503483-cd7d-4392-93b5-4b8a28ed4996 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:05.276919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 230/451] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:24:05 2022] GET /identity/v3/groups/b8324c97af7b4541b768ec0f4a9ae903 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:05.278601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a32fa-cca8-4608-9ef3-cc52653130a0 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:05.283358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2503483-cd7d-4392-93b5-4b8a28ed4996 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:05.312382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb97f4-399c-4d57-bced-dfebf143b362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:05.313602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb97f4-399c-4d57-bced-dfebf143b362 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:05.313945 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb97f4-399c-4d57-bced-dfebf143b362 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:05.314223 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb97f4-399c-4d57-bced-dfebf143b362 None admin] PATH_INFO: `/v3/projects/2c317c5acb304c5d983c73b9ed1b6802` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:05.315367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb97f4-399c-4d57-bced-dfebf143b362 None admin] RBAC: Authorizing `identity:get_project(project_id=2c317c5acb304c5d983c73b9ed1b6802)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:05.316811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb97f4-399c-4d57-bced-dfebf143b362 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:05.318451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 222/452] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:24:05 2022] GET /identity/v3/projects/2c317c5acb304c5d983c73b9ed1b6802 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:05.323852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb97f4-399c-4d57-bced-dfebf143b362 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:05.382241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2503483-cd7d-4392-93b5-4b8a28ed4996 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:05.384023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:05.415188 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d21b51-52c4-4c0c-934f-be39b2386bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:05.415779 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d21b51-52c4-4c0c-934f-be39b2386bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:05.416046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d21b51-52c4-4c0c-934f-be39b2386bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:05.416309 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d21b51-52c4-4c0c-934f-be39b2386bb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:05.417010 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d21b51-52c4-4c0c-934f-be39b2386bb1 None admin] RBAC: Adding query filter params (group.id=b8324c97af7b4541b768ec0f4a9ae903, role.id=93f699f03f0f41498415ab9533bdfcb4, scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:05.417307 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d21b51-52c4-4c0c-934f-be39b2386bb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:05.418744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d21b51-52c4-4c0c-934f-be39b2386bb1 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:05.421029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb97f4-399c-4d57-bced-dfebf143b362 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:05.424387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 231/453] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 16 05:24:05 2022] GET /identity/v3/role_assignments?group.id=b8324c97af7b4541b768ec0f4a9ae903&scope.project.id=082a3596b7724c4f8f59b2687c2ca0fe&role.id=93f699f03f0f41498415ab9533bdfcb4 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:05.457223 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470d0a7-5396-4364-9c10-dd9d231b4382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:05.457892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470d0a7-5396-4364-9c10-dd9d231b4382 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:05.458171 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470d0a7-5396-4364-9c10-dd9d231b4382 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:05.458440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470d0a7-5396-4364-9c10-dd9d231b4382 None admin] PATH_INFO: `/v3/groups/d6bee4d7fb124734a8168a39e29d938d` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:05.459616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470d0a7-5396-4364-9c10-dd9d231b4382 None admin] RBAC: Authorizing `identity:get_group(group_id=d6bee4d7fb124734a8168a39e29d938d)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:05.461068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470d0a7-5396-4364-9c10-dd9d231b4382 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:05.462891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 223/454] 10.222.0.49 () {56 vars in 1180 bytes} [Fri Sep 16 05:24:05 2022] GET /identity/v3/groups/d6bee4d7fb124734a8168a39e29d938d => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:05.469023 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d21b51-52c4-4c0c-934f-be39b2386bb1 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:05.487415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:05.576373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d21b51-52c4-4c0c-934f-be39b2386bb1 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:05.610920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495c0df-5008-4780-9e1c-ef9117ac2829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:05.611525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495c0df-5008-4780-9e1c-ef9117ac2829 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:05.611801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495c0df-5008-4780-9e1c-ef9117ac2829 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:05.612076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495c0df-5008-4780-9e1c-ef9117ac2829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:05.612797 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495c0df-5008-4780-9e1c-ef9117ac2829 None admin] RBAC: Adding query filter params (group.id=d6bee4d7fb124734a8168a39e29d938d, role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=2c317c5acb304c5d983c73b9ed1b6802) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:05.613104 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495c0df-5008-4780-9e1c-ef9117ac2829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:05.614565 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6495c0df-5008-4780-9e1c-ef9117ac2829 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:05.620407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 232/455] 10.222.0.49 () {56 vars in 1376 bytes} [Fri Sep 16 05:24:05 2022] GET /identity/v3/role_assignments?group.id=d6bee4d7fb124734a8168a39e29d938d&scope.project.id=2c317c5acb304c5d983c73b9ed1b6802&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:08.261325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8fe51-18b3-49a2-8550-e62c96996a97 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:08.261325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8fe51-18b3-49a2-8550-e62c96996a97 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:08.261325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc8fe51-18b3-49a2-8550-e62c96996a97 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:08.262226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 224/456] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:08.268020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfd5d5-2dec-464a-bf54-b18c9dc78c38 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:08.268838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfd5d5-2dec-464a-bf54-b18c9dc78c38 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:08.269324 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfd5d5-2dec-464a-bf54-b18c9dc78c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:08.293988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-01bfd5d5-2dec-464a-bf54-b18c9dc78c38 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:08.295500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-01bfd5d5-2dec-464a-bf54-b18c9dc78c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:08.395451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 233/457] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:08.574429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a2012-4282-476e-b000-46fa0f0b17b2 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:08.574626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a2012-4282-476e-b000-46fa0f0b17b2 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:08.575048 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a2012-4282-476e-b000-46fa0f0b17b2 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:08.576759 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 225/458] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:08.582239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7110c38-468c-4d30-a4f7-17ab63d756b0 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:08.582674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7110c38-468c-4d30-a4f7-17ab63d756b0 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:08.583106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7110c38-468c-4d30-a4f7-17ab63d756b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:08.618085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-d7110c38-468c-4d30-a4f7-17ab63d756b0 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:08.619482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d7110c38-468c-4d30-a4f7-17ab63d756b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:08.621685 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:08.712022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 234/459] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:24:08.723630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951c419-42e7-4ddd-a50d-0776ab319434 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:08.723910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951c419-42e7-4ddd-a50d-0776ab319434 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:08.724175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951c419-42e7-4ddd-a50d-0776ab319434 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:08.725694 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 226/460] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:08.731020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bec2f4-947f-4b35-9f4b-1091c0997e60 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:08.731301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bec2f4-947f-4b35-9f4b-1091c0997e60 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:08.731572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bec2f4-947f-4b35-9f4b-1091c0997e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:08.754215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-36bec2f4-947f-4b35-9f4b-1091c0997e60 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:08.755398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-36bec2f4-947f-4b35-9f4b-1091c0997e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:08.758842 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951c419-42e7-4ddd-a50d-0776ab319434 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:08.885008 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 235/461] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:08.917935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951c419-42e7-4ddd-a50d-0776ab319434 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:08.959724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:08.968285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621436a-8363-4929-9f8c-7629985f148f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:08.969700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621436a-8363-4929-9f8c-7629985f148f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:08.971352 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621436a-8363-4929-9f8c-7629985f148f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:08.971352 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621436a-8363-4929-9f8c-7629985f148f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:08.981487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621436a-8363-4929-9f8c-7629985f148f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:08.983257 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621436a-8363-4929-9f8c-7629985f148f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-1621436a-8363-4929-9f8c-7629985f148f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:09.000601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:09.001687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:09.002567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:09.002567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:09.002567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:09.006310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 227/462] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:09.008399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bec2f4-947f-4b35-9f4b-1091c0997e60 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:09.091508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:09.112084 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efa9b4-47ec-495f-bba1-7657628683ea None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:09.112709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efa9b4-47ec-495f-bba1-7657628683ea None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:09.113226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efa9b4-47ec-495f-bba1-7657628683ea None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:09.117924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 228/463] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:24:09.118822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efa9b4-47ec-495f-bba1-7657628683ea None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:09.130192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bec2f4-947f-4b35-9f4b-1091c0997e60 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:09.166337 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2455b-cc82-47c8-92b2-a94c9115481d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:09.167266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2455b-cc82-47c8-92b2-a94c9115481d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:09.167755 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2455b-cc82-47c8-92b2-a94c9115481d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:09.168270 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2455b-cc82-47c8-92b2-a94c9115481d None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:09.169109 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2455b-cc82-47c8-92b2-a94c9115481d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:09.169668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2455b-cc82-47c8-92b2-a94c9115481d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:09.171433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b2455b-cc82-47c8-92b2-a94c9115481d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:09.186452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 236/464] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:09.189191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2768c-d427-44d9-8505-1aa56acd060d None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:09.189683 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2768c-d427-44d9-8505-1aa56acd060d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:09.190518 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2768c-d427-44d9-8505-1aa56acd060d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:09.213164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-02b2768c-d427-44d9-8505-1aa56acd060d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:09.214486 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-02b2768c-d427-44d9-8505-1aa56acd060d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:09.233300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efa9b4-47ec-495f-bba1-7657628683ea None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:09.270337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fe4ca-803b-478f-ba93-69760df00fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:09.270990 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fe4ca-803b-478f-ba93-69760df00fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:09.271283 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fe4ca-803b-478f-ba93-69760df00fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:09.271538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fe4ca-803b-478f-ba93-69760df00fd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:09.282254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 237/465] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:09.283769 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fe4ca-803b-478f-ba93-69760df00fd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:09.285440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fe4ca-803b-478f-ba93-69760df00fd0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:09.288782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2768c-d427-44d9-8505-1aa56acd060d None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-428fe4ca-803b-478f-ba93-69760df00fd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:09.303908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:09.304869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:09.310844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:09.310844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:09.310844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:09.310844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 229/466] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:09.310844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fe4ca-803b-478f-ba93-69760df00fd0 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:09.344633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:09.363852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:09.410607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2768c-d427-44d9-8505-1aa56acd060d None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:09.418004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fe4ca-803b-478f-ba93-69760df00fd0 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:09.449436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cce9a-7800-4f59-885d-491f817f67cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:09.450415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cce9a-7800-4f59-885d-491f817f67cb None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:09.450941 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cce9a-7800-4f59-885d-491f817f67cb None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:09.451503 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cce9a-7800-4f59-885d-491f817f67cb None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:09.452369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31cce9a-7800-4f59-885d-491f817f67cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:09.454686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31cce9a-7800-4f59-885d-491f817f67cb None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:09.456688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f0b88-2b20-43e3-a912-43183b2d4575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:09.457310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f0b88-2b20-43e3-a912-43183b2d4575 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:09.457579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f0b88-2b20-43e3-a912-43183b2d4575 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:09.457849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f0b88-2b20-43e3-a912-43183b2d4575 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:09.470468 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f0b88-2b20-43e3-a912-43183b2d4575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:09.471905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803f0b88-2b20-43e3-a912-43183b2d4575 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-803f0b88-2b20-43e3-a912-43183b2d4575 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:09.487064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:09.487959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:09.488835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:09.488835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:09.488835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:09.490153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 230/467] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:09.491267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f0b88-2b20-43e3-a912-43183b2d4575 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:09.555062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 238/468] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:24:09 2022] POST /identity/v3/users => generated 246 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:24:09.556349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cce9a-7800-4f59-885d-491f817f67cb None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:09.597514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:09.616443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f0b88-2b20-43e3-a912-43183b2d4575 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:09.663407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18301-3e06-4667-8810-99c74d710583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:09.664598 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18301-3e06-4667-8810-99c74d710583 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:09.664991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18301-3e06-4667-8810-99c74d710583 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:09.665364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd18301-3e06-4667-8810-99c74d710583 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:09.667245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd18301-3e06-4667-8810-99c74d710583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:09.667666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd18301-3e06-4667-8810-99c74d710583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:09.669241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd18301-3e06-4667-8810-99c74d710583 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:09.697025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 231/469] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:09.701946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18301-3e06-4667-8810-99c74d710583 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:09.714106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cce9a-7800-4f59-885d-491f817f67cb None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:09.732021 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:09.762982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f1449-b93a-46ae-8fe0-4ad03d116272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:09.763632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f1449-b93a-46ae-8fe0-4ad03d116272 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:09.763892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f1449-b93a-46ae-8fe0-4ad03d116272 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:09.764161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f1449-b93a-46ae-8fe0-4ad03d116272 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:09.764825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f1449-b93a-46ae-8fe0-4ad03d116272 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:09.765122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f1449-b93a-46ae-8fe0-4ad03d116272 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:09.769266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f1449-b93a-46ae-8fe0-4ad03d116272 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:09.793872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 239/470] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:09.793872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f1449-b93a-46ae-8fe0-4ad03d116272 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:09.845859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd18301-3e06-4667-8810-99c74d710583 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:09.887603 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d7885-a880-43d5-9b1f-4599985a7763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:09.888235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d7885-a880-43d5-9b1f-4599985a7763 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:09.888508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d7885-a880-43d5-9b1f-4599985a7763 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:09.888775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d7885-a880-43d5-9b1f-4599985a7763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:09.901067 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d7885-a880-43d5-9b1f-4599985a7763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:09.902547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d7885-a880-43d5-9b1f-4599985a7763 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:09.913647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-bf6d7885-a880-43d5-9b1f-4599985a7763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:09.921917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:09.923615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:09.924671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:09.924671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:09.924671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:09.925006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 232/471] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:09.926084 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d7885-a880-43d5-9b1f-4599985a7763 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:09.939862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f1449-b93a-46ae-8fe0-4ad03d116272 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:09.977457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc005d7b-ca00-4cd3-9726-0e33ae6b1aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:09.978118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc005d7b-ca00-4cd3-9726-0e33ae6b1aa3 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:09.978393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc005d7b-ca00-4cd3-9726-0e33ae6b1aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:09.978659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc005d7b-ca00-4cd3-9726-0e33ae6b1aa3 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:09.979296 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc005d7b-ca00-4cd3-9726-0e33ae6b1aa3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:09.980733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc005d7b-ca00-4cd3-9726-0e33ae6b1aa3 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:10.036624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d7885-a880-43d5-9b1f-4599985a7763 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:10.068996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 240/472] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:24:09 2022] POST /identity/v3/users => generated 248 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:24:10.068996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f4a04-1d47-4c23-9a56-28e9724f6f05 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:10.068996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f4a04-1d47-4c23-9a56-28e9724f6f05 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:10.068996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f4a04-1d47-4c23-9a56-28e9724f6f05 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:10.068996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 241/473] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:10.069632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f4a04-1d47-4c23-9a56-28e9724f6f05 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:10.089028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2a5de-d8a3-4a4a-a810-4655f1376f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:10.090802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2a5de-d8a3-4a4a-a810-4655f1376f7d None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:10.091240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2a5de-d8a3-4a4a-a810-4655f1376f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:10.091668 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2a5de-d8a3-4a4a-a810-4655f1376f7d None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:10.092988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2a5de-d8a3-4a4a-a810-4655f1376f7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:10.095920 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2a5de-d8a3-4a4a-a810-4655f1376f7d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:10.115516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:10.206888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 233/474] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:24:09 2022] POST /identity/v3/users => generated 248 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:24:10.209319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebf76c-e357-4028-859c-d105d5ce1144 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:10.209588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebf76c-e357-4028-859c-d105d5ce1144 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:10.210552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebf76c-e357-4028-859c-d105d5ce1144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:10.212855 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f4a04-1d47-4c23-9a56-28e9724f6f05 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:10.237692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-6eebf76c-e357-4028-859c-d105d5ce1144 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:10.239127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6eebf76c-e357-4028-859c-d105d5ce1144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:10.263180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b57283-bebf-42f5-9d81-62642cc1ec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:10.264117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b57283-bebf-42f5-9d81-62642cc1ec40 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:10.264632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b57283-bebf-42f5-9d81-62642cc1ec40 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:10.265075 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b57283-bebf-42f5-9d81-62642cc1ec40 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:10.266200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b57283-bebf-42f5-9d81-62642cc1ec40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:10.267533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b57283-bebf-42f5-9d81-62642cc1ec40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:10.269509 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b57283-bebf-42f5-9d81-62642cc1ec40 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:10.291637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 242/475] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:10.298054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b57283-bebf-42f5-9d81-62642cc1ec40 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:10.309572 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:10.332062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 234/476] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:10.410367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:10.421442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b57283-bebf-42f5-9d81-62642cc1ec40 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:10.459319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f98ef7-cd1f-424e-a4da-f5a7a7b1f3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:10.459944 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f98ef7-cd1f-424e-a4da-f5a7a7b1f3a9 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:10.460279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f98ef7-cd1f-424e-a4da-f5a7a7b1f3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:10.460605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f98ef7-cd1f-424e-a4da-f5a7a7b1f3a9 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:10.461314 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f98ef7-cd1f-424e-a4da-f5a7a7b1f3a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:10.462858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f98ef7-cd1f-424e-a4da-f5a7a7b1f3a9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:10.549346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 243/477] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:24:10 2022] POST /identity/v3/users => generated 249 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:24:10.629697 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebf76c-e357-4028-859c-d105d5ce1144 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:10.748374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebf76c-e357-4028-859c-d105d5ce1144 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:10.791684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb331d-c189-4934-a318-d9a10ca0ed4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:10.792525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb331d-c189-4934-a318-d9a10ca0ed4b None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:10.793006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb331d-c189-4934-a318-d9a10ca0ed4b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:10.793407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb331d-c189-4934-a318-d9a10ca0ed4b None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:10.794235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb331d-c189-4934-a318-d9a10ca0ed4b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:10.795847 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb331d-c189-4934-a318-d9a10ca0ed4b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:10.841922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.sql.core [None req-edfb331d-c189-4934-a318-d9a10ca0ed4b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 16 05:24:10.841922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 16 05:24:10.841922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [parameters: {'id': 'd8c71823f0b24b9aaab583deaadcdae8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 16 05:24:10.841922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81556) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-edfb331d-c189-4934-a318-d9a10ca0ed4b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application conn.query(q) Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application result.read() Sep 16 05:24:10.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return ref.to_dict() Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 05:24:10.860953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application next(self.gen) Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application yield resource Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application next(self.gen) Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self.session.rollback() Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self.force_reraise() Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise self.value Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application session.commit() Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 16 05:24:10.862324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._prepare_impl() Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self.session.flush() Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._flush(objects) Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application compat.raise_( Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application flush_context.execute() Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rec.execute(self) Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 16 05:24:10.863190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 05:24:10.865237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application conn.query(q) Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application result.read() Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 05:24:10.867191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [parameters: {'id': 'd8c71823f0b24b9aaab583deaadcdae8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:10.870466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 16 05:24:10.872226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:10.874109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 235/478] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Sep 16 05:24:10 2022] POST /identity/v3/roles => generated 144 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:10.874256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f98ef7-cd1f-424e-a4da-f5a7a7b1f3a9 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:11.011449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f98ef7-cd1f-424e-a4da-f5a7a7b1f3a9 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:11.047309 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23dbc3-c24b-4263-ac6b-3a3ad64d8a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:11.047950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23dbc3-c24b-4263-ac6b-3a3ad64d8a24 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:11.048220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23dbc3-c24b-4263-ac6b-3a3ad64d8a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:11.048485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23dbc3-c24b-4263-ac6b-3a3ad64d8a24 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:11.059115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c23dbc3-c24b-4263-ac6b-3a3ad64d8a24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-6c23dbc3-c24b-4263-ac6b-3a3ad64d8a24 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:11.064601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:11.066749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:11.068585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:11.068585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:11.068585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:11.068585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:11.068585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:11.068585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:11.068585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:11.068585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:11.068585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:11.068585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:11.068585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 16 05:24:11.068585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:11.068585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 244/479] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:24:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:11.075989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb331d-c189-4934-a318-d9a10ca0ed4b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:11.146428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:11.197809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb331d-c189-4934-a318-d9a10ca0ed4b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:11.242448 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb25d58-4877-4138-94b3-adc1442da8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:11.243253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb25d58-4877-4138-94b3-adc1442da8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:11.243619 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb25d58-4877-4138-94b3-adc1442da8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:11.244029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb25d58-4877-4138-94b3-adc1442da8c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:11.244804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb25d58-4877-4138-94b3-adc1442da8c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:11.245197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb25d58-4877-4138-94b3-adc1442da8c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:11.250366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb25d58-4877-4138-94b3-adc1442da8c4 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:11.259039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 236/480] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:12.266783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf2fe2-86a6-4be2-80b4-8bdbceb5fb3a None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:12.267273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf2fe2-86a6-4be2-80b4-8bdbceb5fb3a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:12.267273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf2fe2-86a6-4be2-80b4-8bdbceb5fb3a None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:12.268844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 245/481] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:12.274380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ee4a7-9cba-4b1d-a3d6-8bfd334484da None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:12.274792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ee4a7-9cba-4b1d-a3d6-8bfd334484da None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:12.274930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ee4a7-9cba-4b1d-a3d6-8bfd334484da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:12.293211 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-3d2ee4a7-9cba-4b1d-a3d6-8bfd334484da None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:12.294591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3d2ee4a7-9cba-4b1d-a3d6-8bfd334484da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:12.373566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 237/482] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:12.654393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf2fe2-86a6-4be2-80b4-8bdbceb5fb3a None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:12.664377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:12.799695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf2fe2-86a6-4be2-80b4-8bdbceb5fb3a None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:12.842094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1a2a5-9d15-4d99-a9fc-44d73a331560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:12.843090 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1a2a5-9d15-4d99-a9fc-44d73a331560 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:12.844130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1a2a5-9d15-4d99-a9fc-44d73a331560 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:12.844675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1a2a5-9d15-4d99-a9fc-44d73a331560 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:12.856304 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1a2a5-9d15-4d99-a9fc-44d73a331560 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-5cb1a2a5-9d15-4d99-a9fc-44d73a331560 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:12.861589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:12.862495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:12.863323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:12.863323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:12.863323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:12.863323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:12.863323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:12.863323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:12.863323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:12.863323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:12.863323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:12.863323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:12.863323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:12.863323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:12.865902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 246/483] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:12.871850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ee4a7-9cba-4b1d-a3d6-8bfd334484da None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:12.922856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:12.994049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ee4a7-9cba-4b1d-a3d6-8bfd334484da None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:13.028547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c0db3-383b-4d8a-9892-f1e894ce3369 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:13.028851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c0db3-383b-4d8a-9892-f1e894ce3369 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:13.029084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89c0db3-383b-4d8a-9892-f1e894ce3369 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:13.030707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 247/484] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:13.036809 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f9cb8-7e5c-415f-98e9-1bf9eff47542 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:13.037097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f9cb8-7e5c-415f-98e9-1bf9eff47542 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:13.037366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f9cb8-7e5c-415f-98e9-1bf9eff47542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:13.039596 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ecf63-5f4b-4205-852f-31bbbed630b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:13.040195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ecf63-5f4b-4205-852f-31bbbed630b9 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:13.040476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ecf63-5f4b-4205-852f-31bbbed630b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:13.040724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ecf63-5f4b-4205-852f-31bbbed630b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:13.041370 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ecf63-5f4b-4205-852f-31bbbed630b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:13.041663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ecf63-5f4b-4205-852f-31bbbed630b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:13.043032 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ecf63-5f4b-4205-852f-31bbbed630b9 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:13.057160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-df4f9cb8-7e5c-415f-98e9-1bf9eff47542 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:13.058362 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-df4f9cb8-7e5c-415f-98e9-1bf9eff47542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:13.061097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 238/485] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:13.066590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ecf63-5f4b-4205-852f-31bbbed630b9 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:13.134205 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:13.147942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 248/486] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:24:13.149745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb3145-9b0f-4e9f-a6c7-a3a34ae0527b None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:13.150272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb3145-9b0f-4e9f-a6c7-a3a34ae0527b None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:13.150685 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb3145-9b0f-4e9f-a6c7-a3a34ae0527b None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:13.152529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 249/487] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:13.158390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041577eb-d58c-47d1-8e69-2fd64a66b7ac None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:13.158980 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041577eb-d58c-47d1-8e69-2fd64a66b7ac None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:13.159454 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041577eb-d58c-47d1-8e69-2fd64a66b7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:13.189959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-041577eb-d58c-47d1-8e69-2fd64a66b7ac None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:13.191629 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-041577eb-d58c-47d1-8e69-2fd64a66b7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:13.228448 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ecf63-5f4b-4205-852f-31bbbed630b9 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:13.236987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:13.291847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 250/488] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:24:13.321122 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844fad5-e81c-4793-8ed5-f82b13f0023b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:13.321753 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844fad5-e81c-4793-8ed5-f82b13f0023b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:13.322061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844fad5-e81c-4793-8ed5-f82b13f0023b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:13.322333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844fad5-e81c-4793-8ed5-f82b13f0023b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:13.341516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844fad5-e81c-4793-8ed5-f82b13f0023b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:13.343003 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844fad5-e81c-4793-8ed5-f82b13f0023b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-8844fad5-e81c-4793-8ed5-f82b13f0023b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:13.362479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:13.363425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:13.364285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:13.364285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:13.364285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:13.364285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 239/489] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:24:13.370519 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041577eb-d58c-47d1-8e69-2fd64a66b7ac None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:13.422197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb0d76-c74a-4ddb-8a19-910b984e3939 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:13.423855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb0d76-c74a-4ddb-8a19-910b984e3939 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:13.423855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb0d76-c74a-4ddb-8a19-910b984e3939 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:13.424751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 240/490] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:13.433379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75f383-cbec-448b-8df1-134d02070513 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:13.433379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75f383-cbec-448b-8df1-134d02070513 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:13.433379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75f383-cbec-448b-8df1-134d02070513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:13.453939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0d75f383-cbec-448b-8df1-134d02070513 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:13.453939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0d75f383-cbec-448b-8df1-134d02070513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:13.534231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041577eb-d58c-47d1-8e69-2fd64a66b7ac None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:13.548896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 241/491] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:24:13.550444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75f383-cbec-448b-8df1-134d02070513 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:13.575391 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:13.598542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378bc72-52ac-4199-a5c5-3e7f73f82834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:13.599191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378bc72-52ac-4199-a5c5-3e7f73f82834 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:13.599500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378bc72-52ac-4199-a5c5-3e7f73f82834 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:13.599769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378bc72-52ac-4199-a5c5-3e7f73f82834 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:13.600487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378bc72-52ac-4199-a5c5-3e7f73f82834 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:13.600787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378bc72-52ac-4199-a5c5-3e7f73f82834 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:13.602228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378bc72-52ac-4199-a5c5-3e7f73f82834 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:13.617467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 251/492] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:13.622738 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378bc72-52ac-4199-a5c5-3e7f73f82834 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:13.649931 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:13.667734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d75f383-cbec-448b-8df1-134d02070513 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:13.710973 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d56f02-d0d2-4bd2-8e7b-5223f2f5d0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:13.712205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d56f02-d0d2-4bd2-8e7b-5223f2f5d0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:13.712492 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d56f02-d0d2-4bd2-8e7b-5223f2f5d0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:13.712761 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d56f02-d0d2-4bd2-8e7b-5223f2f5d0c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:13.726376 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d56f02-d0d2-4bd2-8e7b-5223f2f5d0c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-d9d56f02-d0d2-4bd2-8e7b-5223f2f5d0c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:13.732775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:13.733646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:13.735123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:13.735123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:13.735123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:13.735123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:13.735123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:13.735123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:13.735123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:13.735123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:13.735123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:13.735123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:13.735123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:13.735123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:13.738314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 242/493] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:13.739118 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d56f02-d0d2-4bd2-8e7b-5223f2f5d0c9 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:13.756250 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378bc72-52ac-4199-a5c5-3e7f73f82834 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:13.801467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8677519-4945-4c63-b95a-4fbdacf7e902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:13.802420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8677519-4945-4c63-b95a-4fbdacf7e902 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:13.802897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8677519-4945-4c63-b95a-4fbdacf7e902 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:13.803357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8677519-4945-4c63-b95a-4fbdacf7e902 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:13.814140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8677519-4945-4c63-b95a-4fbdacf7e902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-f8677519-4945-4c63-b95a-4fbdacf7e902 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:13.819666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:13.820521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:13.821327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:13.821327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:13.821327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:13.821327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:13.821327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:13.821327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:13.821327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:13.821327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:13.821327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:13.821327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:13.821327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:13.821327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:13.828804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 252/494] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:13.829958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8677519-4945-4c63-b95a-4fbdacf7e902 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:13.867033 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d56f02-d0d2-4bd2-8e7b-5223f2f5d0c9 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:13.878119 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:13.907250 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c77392-2e33-411b-910f-e6bbef6a0be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:13.908163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c77392-2e33-411b-910f-e6bbef6a0be4 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:13.908638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c77392-2e33-411b-910f-e6bbef6a0be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:13.909107 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c77392-2e33-411b-910f-e6bbef6a0be4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:13.918314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c77392-2e33-411b-910f-e6bbef6a0be4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:13.919849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c77392-2e33-411b-910f-e6bbef6a0be4 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-41c77392-2e33-411b-910f-e6bbef6a0be4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:13.933516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:13.934617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:13.935511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:13.935511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:13.935511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:13.935511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:13.935511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:13.935511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:13.935511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:13.935511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:13.935511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:24:13.935511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:13.935511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 243/495] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 16 05:24:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:13.936052 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c77392-2e33-411b-910f-e6bbef6a0be4 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:13.945873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8677519-4945-4c63-b95a-4fbdacf7e902 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:13.950334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:13.981689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd7fa5-1d11-4ea1-afac-93fab53f6c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:13.982350 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd7fa5-1d11-4ea1-afac-93fab53f6c60 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:13.982686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd7fa5-1d11-4ea1-afac-93fab53f6c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:13.983194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd7fa5-1d11-4ea1-afac-93fab53f6c60 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:13.983863 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd7fa5-1d11-4ea1-afac-93fab53f6c60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:13.984197 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd7fa5-1d11-4ea1-afac-93fab53f6c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:13.985565 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd7fa5-1d11-4ea1-afac-93fab53f6c60 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:13.993860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 253/496] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:13.999013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd7fa5-1d11-4ea1-afac-93fab53f6c60 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:14.043051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c77392-2e33-411b-910f-e6bbef6a0be4 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:14.082354 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9c3cb-9885-47bc-8bcf-4190a1b3e297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:14.083313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c3cb-9885-47bc-8bcf-4190a1b3e297 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:14.083595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c3cb-9885-47bc-8bcf-4190a1b3e297 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:14.083865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9c3cb-9885-47bc-8bcf-4190a1b3e297 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:14.084641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9c3cb-9885-47bc-8bcf-4190a1b3e297 None admin] RBAC: Adding query filter params (name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:14.085004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9c3cb-9885-47bc-8bcf-4190a1b3e297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:14.086530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9c3cb-9885-47bc-8bcf-4190a1b3e297 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:14.101411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 244/497] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:14.101411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9c3cb-9885-47bc-8bcf-4190a1b3e297 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:14.106468 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd7fa5-1d11-4ea1-afac-93fab53f6c60 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:14.118181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:14.147591 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f76ef-0dc9-46bd-8356-b1496dfdcc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:14.148524 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f76ef-0dc9-46bd-8356-b1496dfdcc54 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:14.148978 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f76ef-0dc9-46bd-8356-b1496dfdcc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:14.149899 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f76ef-0dc9-46bd-8356-b1496dfdcc54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:14.161517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f76ef-0dc9-46bd-8356-b1496dfdcc54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-f59f76ef-0dc9-46bd-8356-b1496dfdcc54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:14.166883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:14.167773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:14.168593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:14.168593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:14.168593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:14.168593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:14.168593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:14.168593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:14.168593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:14.168593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:14.168593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:14.168593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:14.168593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:14.168593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:14.168593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 254/498] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:14.169271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f76ef-0dc9-46bd-8356-b1496dfdcc54 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:14.181613 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:14.227125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9c3cb-9885-47bc-8bcf-4190a1b3e297 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:14.263420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95792c2-7cf8-437a-8e3b-2625eb7c66f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:14.263941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95792c2-7cf8-437a-8e3b-2625eb7c66f5 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:14.264215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95792c2-7cf8-437a-8e3b-2625eb7c66f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:14.264487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95792c2-7cf8-437a-8e3b-2625eb7c66f5 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:14.265200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95792c2-7cf8-437a-8e3b-2625eb7c66f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:14.265530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95792c2-7cf8-437a-8e3b-2625eb7c66f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:14.267240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95792c2-7cf8-437a-8e3b-2625eb7c66f5 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:14.273925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f76ef-0dc9-46bd-8356-b1496dfdcc54 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:14.303898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 245/499] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 16 05:24:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:14.304715 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95792c2-7cf8-437a-8e3b-2625eb7c66f5 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:14.311748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1c296-eb4d-4d3f-928e-43fe58b1338d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:14.312556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1c296-eb4d-4d3f-928e-43fe58b1338d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:14.313041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1c296-eb4d-4d3f-928e-43fe58b1338d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:14.313395 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1c296-eb4d-4d3f-928e-43fe58b1338d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:14.323841 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1c296-eb4d-4d3f-928e-43fe58b1338d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:14.325272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1c296-eb4d-4d3f-928e-43fe58b1338d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-17e1c296-eb4d-4d3f-928e-43fe58b1338d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:14.339328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:14.340378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:14.341237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:14.341237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:14.341237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:14.341237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 255/500] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:14.341237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1c296-eb4d-4d3f-928e-43fe58b1338d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:14.366898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:14.402711 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95792c2-7cf8-437a-8e3b-2625eb7c66f5 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:14.429449 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:14.438138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83994285-9df8-4562-8832-cbe04b854289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:14.438767 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83994285-9df8-4562-8832-cbe04b854289 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:14.439042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83994285-9df8-4562-8832-cbe04b854289 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:14.439308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83994285-9df8-4562-8832-cbe04b854289 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:14.442484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1c296-eb4d-4d3f-928e-43fe58b1338d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:14.452748 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83994285-9df8-4562-8832-cbe04b854289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:14.454931 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83994285-9df8-4562-8832-cbe04b854289 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-83994285-9df8-4562-8832-cbe04b854289 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:14.470092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:14.470994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:14.472075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:14.472075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:14.472075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:14.472075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 246/501] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:14.472655 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd26f6-8a3a-41b6-b2fe-462335faa62e None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:14.472931 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd26f6-8a3a-41b6-b2fe-462335faa62e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:14.473199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd26f6-8a3a-41b6-b2fe-462335faa62e None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:14.474440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 247/502] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:14.475122 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd26f6-8a3a-41b6-b2fe-462335faa62e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:14.480642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3f1d5-dec0-4972-b001-ad9ff0036239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:14.481248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3f1d5-dec0-4972-b001-ad9ff0036239 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:14.481525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3f1d5-dec0-4972-b001-ad9ff0036239 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:14.481794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3f1d5-dec0-4972-b001-ad9ff0036239 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:14.482485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3f1d5-dec0-4972-b001-ad9ff0036239 None admin] RBAC: Adding query filter params (name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:14.482785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3f1d5-dec0-4972-b001-ad9ff0036239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:14.484837 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3f1d5-dec0-4972-b001-ad9ff0036239 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:14.492949 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 256/503] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:14.493729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3f1d5-dec0-4972-b001-ad9ff0036239 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:14.573006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:14.583615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd26f6-8a3a-41b6-b2fe-462335faa62e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:14.599995 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3f1d5-dec0-4972-b001-ad9ff0036239 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:14.617069 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914eb26-a36e-47d2-9106-f8478f0d961a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:14.617670 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914eb26-a36e-47d2-9106-f8478f0d961a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:14.618124 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914eb26-a36e-47d2-9106-f8478f0d961a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:14.618395 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914eb26-a36e-47d2-9106-f8478f0d961a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:14.623513 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fb8a3-287f-441f-b166-8132a8cccf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:14.624105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fb8a3-287f-441f-b166-8132a8cccf83 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:14.624380 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fb8a3-287f-441f-b166-8132a8cccf83 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:14.624665 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fb8a3-287f-441f-b166-8132a8cccf83 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:14.625291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fb8a3-287f-441f-b166-8132a8cccf83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:14.625585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fb8a3-287f-441f-b166-8132a8cccf83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:14.626984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fb8a3-287f-441f-b166-8132a8cccf83 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:14.638506 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 257/504] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:14.639288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fb8a3-287f-441f-b166-8132a8cccf83 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:14.667208 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:14.688194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914eb26-a36e-47d2-9106-f8478f0d961a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:14.689642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914eb26-a36e-47d2-9106-f8478f0d961a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:14.738568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fb8a3-287f-441f-b166-8132a8cccf83 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:14.772066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a1b26-9f16-4c9f-a54f-0adb57d5fa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:14.772661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a1b26-9f16-4c9f-a54f-0adb57d5fa9f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:14.772938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a1b26-9f16-4c9f-a54f-0adb57d5fa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:14.773209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a1b26-9f16-4c9f-a54f-0adb57d5fa9f None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:14.773840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a1b26-9f16-4c9f-a54f-0adb57d5fa9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:14.774154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a1b26-9f16-4c9f-a54f-0adb57d5fa9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:14.775504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a1b26-9f16-4c9f-a54f-0adb57d5fa9f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:14.785557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 248/505] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:24:14.786898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236afb0-6416-4eb6-89fa-2b8bbb350fed None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:14.787176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236afb0-6416-4eb6-89fa-2b8bbb350fed None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:14.787438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236afb0-6416-4eb6-89fa-2b8bbb350fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:14.788580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 258/506] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:14.789583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a1b26-9f16-4c9f-a54f-0adb57d5fa9f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:14.792382 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:14.808824 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-3236afb0-6416-4eb6-89fa-2b8bbb350fed None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:14.810036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3236afb0-6416-4eb6-89fa-2b8bbb350fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:14.878519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 249/507] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:14.879347 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236afb0-6416-4eb6-89fa-2b8bbb350fed None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:14.895158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:14.917668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a1b26-9f16-4c9f-a54f-0adb57d5fa9f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:14.956999 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57615593-f069-4069-8ca7-f8e1a092d0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:14.957629 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57615593-f069-4069-8ca7-f8e1a092d0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:14.957985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57615593-f069-4069-8ca7-f8e1a092d0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:14.958257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57615593-f069-4069-8ca7-f8e1a092d0d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:14.968752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57615593-f069-4069-8ca7-f8e1a092d0d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:14.970317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57615593-f069-4069-8ca7-f8e1a092d0d5 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-57615593-f069-4069-8ca7-f8e1a092d0d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:14.985056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:14.986297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:14.987148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:14.987148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:14.987148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:14.987148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 259/508] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:14.987148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57615593-f069-4069-8ca7-f8e1a092d0d5 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:15.003475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236afb0-6416-4eb6-89fa-2b8bbb350fed None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:15.039705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4276052-e8cd-4ef5-b375-ec827c853e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:15.040324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4276052-e8cd-4ef5-b375-ec827c853e87 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:15.040595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4276052-e8cd-4ef5-b375-ec827c853e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:15.040857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4276052-e8cd-4ef5-b375-ec827c853e87 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:15.044405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:15.048227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4276052-e8cd-4ef5-b375-ec827c853e87 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:15.049665 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4276052-e8cd-4ef5-b375-ec827c853e87 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-f4276052-e8cd-4ef5-b375-ec827c853e87 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:15.061512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:15.063434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:15.064335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:15.064335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:15.064335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:15.064335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:15.064335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:15.064335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:15.064335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:15.064335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:15.064335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 16 05:24:15.064335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:15.064335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 250/509] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:15.064937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4276052-e8cd-4ef5-b375-ec827c853e87 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:15.105241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57615593-f069-4069-8ca7-f8e1a092d0d5 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:15.139852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33b0d6-270c-445f-acf7-eaadb84463d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:15.140929 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33b0d6-270c-445f-acf7-eaadb84463d3 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:15.141268 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33b0d6-270c-445f-acf7-eaadb84463d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:15.141814 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33b0d6-270c-445f-acf7-eaadb84463d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:15.146254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:15.152639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33b0d6-270c-445f-acf7-eaadb84463d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:15.154121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33b0d6-270c-445f-acf7-eaadb84463d3 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-da33b0d6-270c-445f-acf7-eaadb84463d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:15.168412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:15.169523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:15.170412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:15.170412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:15.170412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:15.170412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 260/510] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:15.170412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33b0d6-270c-445f-acf7-eaadb84463d3 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:15.182812 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4276052-e8cd-4ef5-b375-ec827c853e87 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:15.217364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505ddc6-8384-45eb-bf6d-df7e9edf587d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:15.218030 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505ddc6-8384-45eb-bf6d-df7e9edf587d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:15.218309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505ddc6-8384-45eb-bf6d-df7e9edf587d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:15.218621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505ddc6-8384-45eb-bf6d-df7e9edf587d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:15.225882 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505ddc6-8384-45eb-bf6d-df7e9edf587d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:15.227321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505ddc6-8384-45eb-bf6d-df7e9edf587d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-0505ddc6-8384-45eb-bf6d-df7e9edf587d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:15.238936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:15.239975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:15.241219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:15.241219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:15.241219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:15.241219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:15.241219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:15.241219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:15.241219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:15.241219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:15.241219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 05:24:15.241219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:15.241219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 251/511] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:15.241219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505ddc6-8384-45eb-bf6d-df7e9edf587d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:15.266824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33b0d6-270c-445f-acf7-eaadb84463d3 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:15.273471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:15.299654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb88ce6-5b17-41ef-96bb-fac0223e7810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:15.300248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb88ce6-5b17-41ef-96bb-fac0223e7810 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:15.300528 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb88ce6-5b17-41ef-96bb-fac0223e7810 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:15.300798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb88ce6-5b17-41ef-96bb-fac0223e7810 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:15.301423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb88ce6-5b17-41ef-96bb-fac0223e7810 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:15.301715 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb88ce6-5b17-41ef-96bb-fac0223e7810 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:15.303200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb88ce6-5b17-41ef-96bb-fac0223e7810 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:15.314467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 261/512] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:15.315321 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb88ce6-5b17-41ef-96bb-fac0223e7810 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:15.349178 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505ddc6-8384-45eb-bf6d-df7e9edf587d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:15.379962 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:15.387911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf680c1d-bae3-4f10-bcee-a26ffca77888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:15.388511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf680c1d-bae3-4f10-bcee-a26ffca77888 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:15.388780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf680c1d-bae3-4f10-bcee-a26ffca77888 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:15.389049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf680c1d-bae3-4f10-bcee-a26ffca77888 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:15.389756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf680c1d-bae3-4f10-bcee-a26ffca77888 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:15.390131 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf680c1d-bae3-4f10-bcee-a26ffca77888 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:15.392262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf680c1d-bae3-4f10-bcee-a26ffca77888 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:15.409307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 252/513] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Sep 16 05:24:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:15.410146 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf680c1d-bae3-4f10-bcee-a26ffca77888 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:15.428158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb88ce6-5b17-41ef-96bb-fac0223e7810 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:15.461799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc0f49-f2f2-45e3-8de1-5ff91f8c00cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:15.462444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc0f49-f2f2-45e3-8de1-5ff91f8c00cd None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:15.462714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc0f49-f2f2-45e3-8de1-5ff91f8c00cd None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:15.462979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc0f49-f2f2-45e3-8de1-5ff91f8c00cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:15.473745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc0f49-f2f2-45e3-8de1-5ff91f8c00cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:15.475226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc0f49-f2f2-45e3-8de1-5ff91f8c00cd None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-72bc0f49-f2f2-45e3-8de1-5ff91f8c00cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:15.489934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:15.490878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:15.491742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:15.491742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:15.491742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:15.491742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 262/514] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:15.491959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc0f49-f2f2-45e3-8de1-5ff91f8c00cd None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:15.510044 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:15.512755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf680c1d-bae3-4f10-bcee-a26ffca77888 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:15.548507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61dc98-d37e-4536-834f-fe6decd153d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:15.549103 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61dc98-d37e-4536-834f-fe6decd153d9 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:15.549376 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61dc98-d37e-4536-834f-fe6decd153d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:15.549644 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61dc98-d37e-4536-834f-fe6decd153d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:15.550302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61dc98-d37e-4536-834f-fe6decd153d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:15.550602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61dc98-d37e-4536-834f-fe6decd153d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:15.551948 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61dc98-d37e-4536-834f-fe6decd153d9 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:15.563219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 253/515] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:15.563987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61dc98-d37e-4536-834f-fe6decd153d9 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:15.590850 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc0f49-f2f2-45e3-8de1-5ff91f8c00cd None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:15.610012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:15.623486 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43223257-b041-434e-9440-7019e19bcf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:15.624086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43223257-b041-434e-9440-7019e19bcf67 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:15.624356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43223257-b041-434e-9440-7019e19bcf67 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:15.625202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43223257-b041-434e-9440-7019e19bcf67 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:15.625932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43223257-b041-434e-9440-7019e19bcf67 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:15.626234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43223257-b041-434e-9440-7019e19bcf67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:15.627633 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43223257-b041-434e-9440-7019e19bcf67 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:15.663051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61dc98-d37e-4536-834f-fe6decd153d9 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:15.665766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 263/516] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Sep 16 05:24:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:15.666640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43223257-b041-434e-9440-7019e19bcf67 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:15.698776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a773949-710e-4b1e-baf4-7924c482b2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:15.699420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a773949-710e-4b1e-baf4-7924c482b2bc None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:15.699696 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a773949-710e-4b1e-baf4-7924c482b2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:15.699963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a773949-710e-4b1e-baf4-7924c482b2bc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:15.707344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a773949-710e-4b1e-baf4-7924c482b2bc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:15.708768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a773949-710e-4b1e-baf4-7924c482b2bc None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-8a773949-710e-4b1e-baf4-7924c482b2bc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:15.720940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:15.721850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:15.722801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:15.722801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:15.722801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:15.722801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:15.722801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:15.722801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:15.722801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:15.722801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:15.722801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 05:24:15.722801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:15.722801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 254/517] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:15.722801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a773949-710e-4b1e-baf4-7924c482b2bc None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:15.747254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:15.769388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43223257-b041-434e-9440-7019e19bcf67 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:15.803015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a29300-e5d9-40dc-b750-d747fcb826e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:15.803642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a29300-e5d9-40dc-b750-d747fcb826e0 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:15.803914 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a29300-e5d9-40dc-b750-d747fcb826e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:15.804187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a29300-e5d9-40dc-b750-d747fcb826e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:15.820864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a773949-710e-4b1e-baf4-7924c482b2bc None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:15.827686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:15.853798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cc91d-f71f-44c5-abc4-66cd787af4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:15.854429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cc91d-f71f-44c5-abc4-66cd787af4f3 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:15.854704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cc91d-f71f-44c5-abc4-66cd787af4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:15.854970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cc91d-f71f-44c5-abc4-66cd787af4f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:15.855600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147cc91d-f71f-44c5-abc4-66cd787af4f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:15.855893 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147cc91d-f71f-44c5-abc4-66cd787af4f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:15.857241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147cc91d-f71f-44c5-abc4-66cd787af4f3 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:15.868964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 255/518] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:15.869925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cc91d-f71f-44c5-abc4-66cd787af4f3 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:15.870847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a29300-e5d9-40dc-b750-d747fcb826e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:15.872310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a29300-e5d9-40dc-b750-d747fcb826e0 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:15.947265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:15.969547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cc91d-f71f-44c5-abc4-66cd787af4f3 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:15.970147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 264/519] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:24:15.970991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a29300-e5d9-40dc-b750-d747fcb826e0 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:16.005728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4090a-6359-4a32-b994-789f7d77ff32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:16.006374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4090a-6359-4a32-b994-789f7d77ff32 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:16.006645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4090a-6359-4a32-b994-789f7d77ff32 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:16.006907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a4090a-6359-4a32-b994-789f7d77ff32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:16.013578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a4090a-6359-4a32-b994-789f7d77ff32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:16.014813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a4090a-6359-4a32-b994-789f7d77ff32 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-b0a4090a-6359-4a32-b994-789f7d77ff32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:16.026630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:16.027573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:16.028433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:16.028433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:16.028433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:16.028433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:16.028433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:16.028433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:16.028433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:16.028433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:16.028433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 256/520] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:16.029051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4090a-6359-4a32-b994-789f7d77ff32 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:16.054946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:16.071917 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a29300-e5d9-40dc-b750-d747fcb826e0 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:16.104405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758680ff-71f8-4ea3-859a-96ffab4d0a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:16.104994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758680ff-71f8-4ea3-859a-96ffab4d0a04 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:16.105265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758680ff-71f8-4ea3-859a-96ffab4d0a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:16.105529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758680ff-71f8-4ea3-859a-96ffab4d0a04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:16.127322 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a4090a-6359-4a32-b994-789f7d77ff32 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:16.149153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:16.160931 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6b3c9-dffe-4bc5-8fbe-89f5aba12d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:16.161514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6b3c9-dffe-4bc5-8fbe-89f5aba12d22 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:16.161786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6b3c9-dffe-4bc5-8fbe-89f5aba12d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:16.162076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6b3c9-dffe-4bc5-8fbe-89f5aba12d22 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:16.163001 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6b3c9-dffe-4bc5-8fbe-89f5aba12d22 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:16.163102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6b3c9-dffe-4bc5-8fbe-89f5aba12d22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:16.164519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6b3c9-dffe-4bc5-8fbe-89f5aba12d22 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:16.170636 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758680ff-71f8-4ea3-859a-96ffab4d0a04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:16.172796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758680ff-71f8-4ea3-859a-96ffab4d0a04 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:16.181560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 257/521] 10.222.0.49 () {56 vars in 1139 bytes} [Fri Sep 16 05:24:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:16.182361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6b3c9-dffe-4bc5-8fbe-89f5aba12d22 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:16.271223 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 265/522] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:24:16.272063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758680ff-71f8-4ea3-859a-96ffab4d0a04 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:16.277645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:16.294811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6b3c9-dffe-4bc5-8fbe-89f5aba12d22 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:16.329172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb888b-454e-415e-941b-6b50f5d1f01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:16.329774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb888b-454e-415e-941b-6b50f5d1f01d None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:16.330075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb888b-454e-415e-941b-6b50f5d1f01d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:16.330345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb888b-454e-415e-941b-6b50f5d1f01d None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:16.330976 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb888b-454e-415e-941b-6b50f5d1f01d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:16.332380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb888b-454e-415e-941b-6b50f5d1f01d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:16.347187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:16.368642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758680ff-71f8-4ea3-859a-96ffab4d0a04 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:16.402802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663a330-c5eb-4f97-b9bc-4e498b8b3b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:16.403412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663a330-c5eb-4f97-b9bc-4e498b8b3b66 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:16.403684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663a330-c5eb-4f97-b9bc-4e498b8b3b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:16.403982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663a330-c5eb-4f97-b9bc-4e498b8b3b66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:16.406895 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 258/523] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:24:16 2022] POST /identity/v3/users => generated 247 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:24:16.407718 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb888b-454e-415e-941b-6b50f5d1f01d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:16.414082 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663a330-c5eb-4f97-b9bc-4e498b8b3b66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:16.415563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5663a330-c5eb-4f97-b9bc-4e498b8b3b66 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-5663a330-c5eb-4f97-b9bc-4e498b8b3b66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:16.432938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:16.434063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:16.434994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:16.434994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:16.434994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:16.434994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 266/524] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:16.434994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663a330-c5eb-4f97-b9bc-4e498b8b3b66 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:16.547235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb888b-454e-415e-941b-6b50f5d1f01d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:16.547781 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5663a330-c5eb-4f97-b9bc-4e498b8b3b66 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:16.575020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:16.584503 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024e3cd-d1e0-457a-bbf9-3e1e31265af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:16.585114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024e3cd-d1e0-457a-bbf9-3e1e31265af7 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:16.586124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024e3cd-d1e0-457a-bbf9-3e1e31265af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:16.586124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024e3cd-d1e0-457a-bbf9-3e1e31265af7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:16.588565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac1c45-01a0-48b3-8165-bef636da7b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:16.589214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac1c45-01a0-48b3-8165-bef636da7b69 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:16.589483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac1c45-01a0-48b3-8165-bef636da7b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:16.589746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac1c45-01a0-48b3-8165-bef636da7b69 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:16.590453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac1c45-01a0-48b3-8165-bef636da7b69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:16.590749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac1c45-01a0-48b3-8165-bef636da7b69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:16.592150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac1c45-01a0-48b3-8165-bef636da7b69 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:16.607089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:16.611121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 259/525] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:16.612014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac1c45-01a0-48b3-8165-bef636da7b69 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:16.670725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024e3cd-d1e0-457a-bbf9-3e1e31265af7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:16.672639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024e3cd-d1e0-457a-bbf9-3e1e31265af7 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:16.723982 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac1c45-01a0-48b3-8165-bef636da7b69 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:16.761247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db164d-753d-495e-87db-00dcd4a6c86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:16.761910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db164d-753d-495e-87db-00dcd4a6c86e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:16.762186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db164d-753d-495e-87db-00dcd4a6c86e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:16.762454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db164d-753d-495e-87db-00dcd4a6c86e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:16.775096 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db164d-753d-495e-87db-00dcd4a6c86e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:16.776575 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db164d-753d-495e-87db-00dcd4a6c86e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:16.789774 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 267/526] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:24:16.789774 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024e3cd-d1e0-457a-bbf9-3e1e31265af7 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-64db164d-753d-495e-87db-00dcd4a6c86e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:16.798393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:16.799334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:16.800188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:16.800188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:16.800188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:16.800188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 260/527] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:16.800404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db164d-753d-495e-87db-00dcd4a6c86e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:16.819500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:16.863173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:16.892571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024e3cd-d1e0-457a-bbf9-3e1e31265af7 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:16.917312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db164d-753d-495e-87db-00dcd4a6c86e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:16.928717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067591b-86da-4766-9ba0-9656ba3f2278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:16.929327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067591b-86da-4766-9ba0-9656ba3f2278 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:16.929601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067591b-86da-4766-9ba0-9656ba3f2278 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:16.929890 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067591b-86da-4766-9ba0-9656ba3f2278 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:16.930529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067591b-86da-4766-9ba0-9656ba3f2278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:16.930827 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067591b-86da-4766-9ba0-9656ba3f2278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:16.932182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f067591b-86da-4766-9ba0-9656ba3f2278 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:16.944062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 268/528] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:16.944983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067591b-86da-4766-9ba0-9656ba3f2278 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:16.958453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc9fb-0173-4b0b-9853-4052d534afc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:16.959058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc9fb-0173-4b0b-9853-4052d534afc5 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:16.959334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc9fb-0173-4b0b-9853-4052d534afc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:16.959600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc9fb-0173-4b0b-9853-4052d534afc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:16.960318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfc9fb-0173-4b0b-9853-4052d534afc5 None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=7db75c20e9744feb8b5dc78c554f4977) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:16.960623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfc9fb-0173-4b0b-9853-4052d534afc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:16.962056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbfc9fb-0173-4b0b-9853-4052d534afc5 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:16.967781 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 261/529] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:16 2022] GET /identity/v3/role_assignments?user.id=7db75c20e9744feb8b5dc78c554f4977&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:16.978046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc9fb-0173-4b0b-9853-4052d534afc5 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:17.054031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:17.068056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f067591b-86da-4766-9ba0-9656ba3f2278 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:17.105634 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614d489-2033-4dee-827f-2fd93a544879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:17.106633 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614d489-2033-4dee-827f-2fd93a544879 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:17.107114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614d489-2033-4dee-827f-2fd93a544879 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:17.107534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614d489-2033-4dee-827f-2fd93a544879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:17.108582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc9fb-0173-4b0b-9853-4052d534afc5 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:17.119508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614d489-2033-4dee-827f-2fd93a544879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:17.121398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614d489-2033-4dee-827f-2fd93a544879 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:17.130275 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-0614d489-2033-4dee-827f-2fd93a544879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:17.145616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:17.146649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:17.147528 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:17.147528 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:17.147528 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:17.149876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 269/530] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:17.150893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614d489-2033-4dee-827f-2fd93a544879 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:17.160643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2568c-ab24-46eb-98c7-853bd425cdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:17.161406 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2568c-ab24-46eb-98c7-853bd425cdda None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:17.161783 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2568c-ab24-46eb-98c7-853bd425cdda None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:17.162177 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2568c-ab24-46eb-98c7-853bd425cdda None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:17.162926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2568c-ab24-46eb-98c7-853bd425cdda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:17.163328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2568c-ab24-46eb-98c7-853bd425cdda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:17.165488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2568c-ab24-46eb-98c7-853bd425cdda None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:17.182325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 262/531] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:17.183705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2568c-ab24-46eb-98c7-853bd425cdda None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:17.279009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614d489-2033-4dee-827f-2fd93a544879 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:17.303816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2568c-ab24-46eb-98c7-853bd425cdda None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:17.315054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3f7b4-6d89-467a-b404-682bea2e4a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:17.315681 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3f7b4-6d89-467a-b404-682bea2e4a71 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:17.315962 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3f7b4-6d89-467a-b404-682bea2e4a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:17.316235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3f7b4-6d89-467a-b404-682bea2e4a71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:17.323497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3f7b4-6d89-467a-b404-682bea2e4a71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:17.324941 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3f7b4-6d89-467a-b404-682bea2e4a71 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:17.327521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-efd3f7b4-6d89-467a-b404-682bea2e4a71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:17.338984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:17.341433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:17.342327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:17.342327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:17.342327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:17.342327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:17.342327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:17.342327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:17.342327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:17.342327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:17.344955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 270/532] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:17.345897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3f7b4-6d89-467a-b404-682bea2e4a71 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:17.353102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73f5bf-71d2-4010-9c3a-23818a08d776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:17.354160 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73f5bf-71d2-4010-9c3a-23818a08d776 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:17.354635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73f5bf-71d2-4010-9c3a-23818a08d776 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:17.355118 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73f5bf-71d2-4010-9c3a-23818a08d776 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:17.355975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73f5bf-71d2-4010-9c3a-23818a08d776 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:17.357163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73f5bf-71d2-4010-9c3a-23818a08d776 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:17.359517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73f5bf-71d2-4010-9c3a-23818a08d776 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:17.376610 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 263/533] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:17.377855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73f5bf-71d2-4010-9c3a-23818a08d776 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:17.421638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:17.470247 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3f7b4-6d89-467a-b404-682bea2e4a71 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:17.507536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73f5bf-71d2-4010-9c3a-23818a08d776 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:17.513339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb105de3-e159-4937-a6ae-c7398185861d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:17.514006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb105de3-e159-4937-a6ae-c7398185861d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:17.514282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb105de3-e159-4937-a6ae-c7398185861d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:17.514549 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb105de3-e159-4937-a6ae-c7398185861d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:17.523856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb105de3-e159-4937-a6ae-c7398185861d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:17.525566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb105de3-e159-4937-a6ae-c7398185861d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-cb105de3-e159-4937-a6ae-c7398185861d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:17.540950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:17.541784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:17.542663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:17.542663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:17.542663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:17.542663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:17.542663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:17.542663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:17.542663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:17.542663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:17.542663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 271/534] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:17.543203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb105de3-e159-4937-a6ae-c7398185861d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:17.548467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ecea1-f220-4e17-8a42-9e0928a75d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:17.549080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ecea1-f220-4e17-8a42-9e0928a75d1b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:17.549349 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ecea1-f220-4e17-8a42-9e0928a75d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:17.549612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ecea1-f220-4e17-8a42-9e0928a75d1b None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:17.550334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ecea1-f220-4e17-8a42-9e0928a75d1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:17.550630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ecea1-f220-4e17-8a42-9e0928a75d1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:17.552109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ecea1-f220-4e17-8a42-9e0928a75d1b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:17.566291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 264/535] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:17.567168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ecea1-f220-4e17-8a42-9e0928a75d1b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:17.611039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:17.657137 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb105de3-e159-4937-a6ae-c7398185861d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:17.676300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:17.683421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ecea1-f220-4e17-8a42-9e0928a75d1b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:17.700991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae245c-a92f-44ba-b20b-5c2206ffb019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:17.701750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae245c-a92f-44ba-b20b-5c2206ffb019 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:17.702084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae245c-a92f-44ba-b20b-5c2206ffb019 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:17.702353 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae245c-a92f-44ba-b20b-5c2206ffb019 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:17.711347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae245c-a92f-44ba-b20b-5c2206ffb019 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:17.713536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae245c-a92f-44ba-b20b-5c2206ffb019 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:17.720138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202444f4-0fca-43af-93f1-65c1bfbfd7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:17.720743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202444f4-0fca-43af-93f1-65c1bfbfd7ca None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:17.721015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202444f4-0fca-43af-93f1-65c1bfbfd7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:17.721285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202444f4-0fca-43af-93f1-65c1bfbfd7ca None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:17.722005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202444f4-0fca-43af-93f1-65c1bfbfd7ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:17.722302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202444f4-0fca-43af-93f1-65c1bfbfd7ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:17.723766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202444f4-0fca-43af-93f1-65c1bfbfd7ca None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-3fae245c-a92f-44ba-b20b-5c2206ffb019 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:17.727317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:17.728166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:17.729013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:17.729013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:17.729013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:17.729013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:17.729013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:17.729013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:17.729013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:17.729013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:17.729013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 272/536] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:17.729480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae245c-a92f-44ba-b20b-5c2206ffb019 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:17.737810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 265/537] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:17.738620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202444f4-0fca-43af-93f1-65c1bfbfd7ca None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:17.853575 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202444f4-0fca-43af-93f1-65c1bfbfd7ca None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:17.877605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae245c-a92f-44ba-b20b-5c2206ffb019 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:17.889476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37dc83c-988b-4b19-80f1-9d717a54187d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:17.890122 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37dc83c-988b-4b19-80f1-9d717a54187d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:17.890452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37dc83c-988b-4b19-80f1-9d717a54187d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:17.890700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37dc83c-988b-4b19-80f1-9d717a54187d None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:17.891412 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37dc83c-988b-4b19-80f1-9d717a54187d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:17.891971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37dc83c-988b-4b19-80f1-9d717a54187d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:17.893430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37dc83c-988b-4b19-80f1-9d717a54187d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:17.906390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:17.908148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 266/538] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:17.908592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37dc83c-988b-4b19-80f1-9d717a54187d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:17.925251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9cca6-a4ed-4aa0-920b-fe4f10165b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:17.926185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9cca6-a4ed-4aa0-920b-fe4f10165b91 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:17.926691 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9cca6-a4ed-4aa0-920b-fe4f10165b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:17.927218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9cca6-a4ed-4aa0-920b-fe4f10165b91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:17.928684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9cca6-a4ed-4aa0-920b-fe4f10165b91 None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=30d96ec6235843c5b67d87a3d0ff0f8f) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:17.929197 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9cca6-a4ed-4aa0-920b-fe4f10165b91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:17.931301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9cca6-a4ed-4aa0-920b-fe4f10165b91 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:17.938798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:17.941231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 273/539] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:17 2022] GET /identity/v3/role_assignments?user.id=30d96ec6235843c5b67d87a3d0ff0f8f&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:17.942375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9cca6-a4ed-4aa0-920b-fe4f10165b91 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:18.034878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37dc83c-988b-4b19-80f1-9d717a54187d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:18.077181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9cca6-a4ed-4aa0-920b-fe4f10165b91 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:18.091568 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44cafb-3abb-43b0-820f-34c8fc60da2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:18.092192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44cafb-3abb-43b0-820f-34c8fc60da2a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:18.092462 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44cafb-3abb-43b0-820f-34c8fc60da2a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:18.092728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44cafb-3abb-43b0-820f-34c8fc60da2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:18.094061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44cafb-3abb-43b0-820f-34c8fc60da2a None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=ff50e50587254bac912fb342a7e09891) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:18.094439 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44cafb-3abb-43b0-820f-34c8fc60da2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:18.095932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44cafb-3abb-43b0-820f-34c8fc60da2a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:18.110371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 267/540] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:17 2022] GET /identity/v3/role_assignments?user.id=ff50e50587254bac912fb342a7e09891&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:18.143263 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22959c0-41b6-4d85-9d48-119f5eaf2afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:18.143893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22959c0-41b6-4d85-9d48-119f5eaf2afa None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:18.144168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22959c0-41b6-4d85-9d48-119f5eaf2afa None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:18.144436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22959c0-41b6-4d85-9d48-119f5eaf2afa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:18.145255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22959c0-41b6-4d85-9d48-119f5eaf2afa None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=3e923be9aeed43728690d24d85dbe062) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:18.145563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22959c0-41b6-4d85-9d48-119f5eaf2afa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:18.147123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22959c0-41b6-4d85-9d48-119f5eaf2afa None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:18.157916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 274/541] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:17 2022] GET /identity/v3/role_assignments?user.id=3e923be9aeed43728690d24d85dbe062&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:18.814071 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203120ea-f4f3-4e4d-a1e0-81ce3462ec78 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:18.814581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203120ea-f4f3-4e4d-a1e0-81ce3462ec78 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:18.814581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203120ea-f4f3-4e4d-a1e0-81ce3462ec78 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:18.816273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 268/542] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:18.824179 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a554f-23c6-485a-8fb3-c58b5d88727f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:18.824619 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a554f-23c6-485a-8fb3-c58b5d88727f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:18.824805 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a554f-23c6-485a-8fb3-c58b5d88727f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:18.849182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-189a554f-23c6-485a-8fb3-c58b5d88727f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:18.850406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-189a554f-23c6-485a-8fb3-c58b5d88727f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:18.908810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:18.945318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 275/543] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:19.332956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203120ea-f4f3-4e4d-a1e0-81ce3462ec78 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:19.444086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:19.462724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb628f47-69ec-4761-86b8-35abee57ddc8 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:19.467306 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb628f47-69ec-4761-86b8-35abee57ddc8 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:19.467306 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb628f47-69ec-4761-86b8-35abee57ddc8 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:19.467306 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 276/544] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:24:19.470120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203120ea-f4f3-4e4d-a1e0-81ce3462ec78 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:19.470247 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7567-170d-4efe-a460-45a67efab25a None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:19.470451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7567-170d-4efe-a460-45a67efab25a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:19.470670 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7567-170d-4efe-a460-45a67efab25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:19.507989 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-ecbe7567-170d-4efe-a460-45a67efab25a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:19.509296 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ecbe7567-170d-4efe-a460-45a67efab25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:19.512076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fa585-3966-4458-b6e9-cbf78b156377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:19.513558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fa585-3966-4458-b6e9-cbf78b156377 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:19.513856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fa585-3966-4458-b6e9-cbf78b156377 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:19.514136 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0fa585-3966-4458-b6e9-cbf78b156377 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:19.530590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0fa585-3966-4458-b6e9-cbf78b156377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-9e0fa585-3966-4458-b6e9-cbf78b156377 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:19.540358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:19.541287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:19.542132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:19.542132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:19.542132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:19.542132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:19.542132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:19.542132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:19.542132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:19.542132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:19.542132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:19.542132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:19.542132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:19.542132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:19.542132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 269/545] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:19.549509 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fa585-3966-4458-b6e9-cbf78b156377 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:19.628463 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 277/546] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:24:19.715484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0fa585-3966-4458-b6e9-cbf78b156377 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:19.764434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a166a4-3be6-46e6-8d70-416d723834f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:19.765029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a166a4-3be6-46e6-8d70-416d723834f4 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:19.765845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a166a4-3be6-46e6-8d70-416d723834f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:19.766117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a166a4-3be6-46e6-8d70-416d723834f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:19.767013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a166a4-3be6-46e6-8d70-416d723834f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:19.767303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a166a4-3be6-46e6-8d70-416d723834f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:19.768652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a166a4-3be6-46e6-8d70-416d723834f4 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:19.779354 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 270/547] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:19.785551 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe7567-170d-4efe-a460-45a67efab25a None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:19.889095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe7567-170d-4efe-a460-45a67efab25a None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:19.899175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:19.924325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb72da-bfc5-4ad4-80be-77b2a95eef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:19.929845 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb72da-bfc5-4ad4-80be-77b2a95eef8e None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:19.929845 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb72da-bfc5-4ad4-80be-77b2a95eef8e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:19.929845 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb72da-bfc5-4ad4-80be-77b2a95eef8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:19.935574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb72da-bfc5-4ad4-80be-77b2a95eef8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:19.937137 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb72da-bfc5-4ad4-80be-77b2a95eef8e None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-cabb72da-bfc5-4ad4-80be-77b2a95eef8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:19.952051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:19.953669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:19.955045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:19.955045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:19.955045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:19.955045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 278/548] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:19.963006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a166a4-3be6-46e6-8d70-416d723834f4 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:19.970972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:20.048340 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb72da-bfc5-4ad4-80be-77b2a95eef8e None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:20.143356 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a166a4-3be6-46e6-8d70-416d723834f4 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:20.196876 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763139d-901d-4b93-888c-690ce4c53bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:20.197481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763139d-901d-4b93-888c-690ce4c53bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:20.198093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763139d-901d-4b93-888c-690ce4c53bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:20.198418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763139d-901d-4b93-888c-690ce4c53bd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:20.199116 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763139d-901d-4b93-888c-690ce4c53bd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:20.199482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763139d-901d-4b93-888c-690ce4c53bd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:20.200895 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763139d-901d-4b93-888c-690ce4c53bd0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:20.214467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 271/549] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:20.221500 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763139d-901d-4b93-888c-690ce4c53bd0 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:20.229369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb72da-bfc5-4ad4-80be-77b2a95eef8e None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:20.310107 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5dd60-c435-4f89-95b9-2666efa22b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:20.310732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5dd60-c435-4f89-95b9-2666efa22b36 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:20.311003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5dd60-c435-4f89-95b9-2666efa22b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:20.311319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5dd60-c435-4f89-95b9-2666efa22b36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:20.321975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5dd60-c435-4f89-95b9-2666efa22b36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:20.322512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-f6f5dd60-c435-4f89-95b9-2666efa22b36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:20.327480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:20.328478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:20.329280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:20.329280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:20.329280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:20.329280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:20.329280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:20.329280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:20.329280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:20.329280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:20.329280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:20.329280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:20.329280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:20.329280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:20.329280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 279/550] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:20.334699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5dd60-c435-4f89-95b9-2666efa22b36 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:20.366221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:20.397429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763139d-901d-4b93-888c-690ce4c53bd0 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:20.456717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee4b6d-4886-4638-9a27-8d13deb149d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:20.457359 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee4b6d-4886-4638-9a27-8d13deb149d6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:20.457633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee4b6d-4886-4638-9a27-8d13deb149d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:20.457929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee4b6d-4886-4638-9a27-8d13deb149d6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:20.474721 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee4b6d-4886-4638-9a27-8d13deb149d6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:20.476200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee4b6d-4886-4638-9a27-8d13deb149d6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-20ee4b6d-4886-4638-9a27-8d13deb149d6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:20.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:20.497456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:20.498417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:20.498417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:20.498417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:20.498417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:20.498417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:20.498417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:20.498417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:20.498417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:20.498417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 05:24:20.498417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:20.498417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 272/551] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:24:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:20.510559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee4b6d-4886-4638-9a27-8d13deb149d6 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:20.520785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5dd60-c435-4f89-95b9-2666efa22b36 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:20.583422 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9b258-43d0-4809-b313-56774556420a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:20.584038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9b258-43d0-4809-b313-56774556420a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:20.584310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9b258-43d0-4809-b313-56774556420a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:20.584577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9b258-43d0-4809-b313-56774556420a None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:20.585242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9b258-43d0-4809-b313-56774556420a None admin] RBAC: Adding query filter params (name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:20.585537 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9b258-43d0-4809-b313-56774556420a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:20.587054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9b258-43d0-4809-b313-56774556420a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:20.601982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 280/552] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:20.604657 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5665a-2329-4d21-bbff-5df9b771487b None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:20.604933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5665a-2329-4d21-bbff-5df9b771487b None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:20.605202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5665a-2329-4d21-bbff-5df9b771487b None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:20.606652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 281/553] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:20.607414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5665a-2329-4d21-bbff-5df9b771487b None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:20.670107 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee4b6d-4886-4638-9a27-8d13deb149d6 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:20.698663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:20.711293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baabdb8d-6db7-4997-8832-c5c34d68af74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:20.712153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabdb8d-6db7-4997-8832-c5c34d68af74 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:20.714167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabdb8d-6db7-4997-8832-c5c34d68af74 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:20.714167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baabdb8d-6db7-4997-8832-c5c34d68af74 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:20.714405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:20.715053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabdb8d-6db7-4997-8832-c5c34d68af74 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:20.715549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabdb8d-6db7-4997-8832-c5c34d68af74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:20.717147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baabdb8d-6db7-4997-8832-c5c34d68af74 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:20.734470 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5665a-2329-4d21-bbff-5df9b771487b None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:20.741229 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 273/554] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Sep 16 05:24:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:20.742865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8012c0a-41db-4f41-b1f9-1cb469417c09 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:20.743308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8012c0a-41db-4f41-b1f9-1cb469417c09 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:20.743868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8012c0a-41db-4f41-b1f9-1cb469417c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:20.764276 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-b8012c0a-41db-4f41-b1f9-1cb469417c09 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:20.765959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b8012c0a-41db-4f41-b1f9-1cb469417c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:20.773556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c380e42-4cb2-4eaa-b7de-40d5dde60792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:20.774215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c380e42-4cb2-4eaa-b7de-40d5dde60792 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:20.774488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c380e42-4cb2-4eaa-b7de-40d5dde60792 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:20.774751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c380e42-4cb2-4eaa-b7de-40d5dde60792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:20.786271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c380e42-4cb2-4eaa-b7de-40d5dde60792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:20.787813 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c380e42-4cb2-4eaa-b7de-40d5dde60792 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-5c380e42-4cb2-4eaa-b7de-40d5dde60792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:20.805158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:20.806162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:20.807131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:20.807131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:20.807131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:20.807131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 282/555] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:20.807890 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c380e42-4cb2-4eaa-b7de-40d5dde60792 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:20.849085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 274/556] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:20.849085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833891e-f731-4af5-8bcf-5490bfc3d305 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:20.849085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833891e-f731-4af5-8bcf-5490bfc3d305 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:20.849085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833891e-f731-4af5-8bcf-5490bfc3d305 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:20.849085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 275/557] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:20.849085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833891e-f731-4af5-8bcf-5490bfc3d305 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:20.929097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c380e42-4cb2-4eaa-b7de-40d5dde60792 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:20.970770 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0407d-2b14-4773-a5a0-d32a67a2ba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:20.971501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0407d-2b14-4773-a5a0-d32a67a2ba92 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:20.971773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0407d-2b14-4773-a5a0-d32a67a2ba92 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:20.972131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0407d-2b14-4773-a5a0-d32a67a2ba92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:20.975060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:20.982556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833891e-f731-4af5-8bcf-5490bfc3d305 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:20.994969 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:21.026958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562c0bd-ad69-401c-9e37-5b767049cdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:21.027809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562c0bd-ad69-401c-9e37-5b767049cdaa None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:21.028087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562c0bd-ad69-401c-9e37-5b767049cdaa None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:21.028354 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562c0bd-ad69-401c-9e37-5b767049cdaa None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:21.029384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562c0bd-ad69-401c-9e37-5b767049cdaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:21.029688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562c0bd-ad69-401c-9e37-5b767049cdaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:21.031791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562c0bd-ad69-401c-9e37-5b767049cdaa None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:21.049982 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 276/558] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:21.051561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7484d-1d26-4642-9cd1-60e686fa8c9e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:21.052050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7484d-1d26-4642-9cd1-60e686fa8c9e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:21.052903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7484d-1d26-4642-9cd1-60e686fa8c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:21.056746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0407d-2b14-4773-a5a0-d32a67a2ba92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:21.058241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0407d-2b14-4773-a5a0-d32a67a2ba92 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:21.074878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-47a7484d-1d26-4642-9cd1-60e686fa8c9e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:21.076113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-47a7484d-1d26-4642-9cd1-60e686fa8c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:21.148537 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 277/559] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:21.149429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7484d-1d26-4642-9cd1-60e686fa8c9e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:21.166446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 283/560] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:24:21.166446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cdb02-be05-4580-bbd6-58abf5aa59e7 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:21.166446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cdb02-be05-4580-bbd6-58abf5aa59e7 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:21.166446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8cdb02-be05-4580-bbd6-58abf5aa59e7 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:21.166446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 284/561] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:21.166446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8cdb02-be05-4580-bbd6-58abf5aa59e7 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:21.201997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:21.259504 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:21.262381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7484d-1d26-4642-9cd1-60e686fa8c9e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:21.271437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8cdb02-be05-4580-bbd6-58abf5aa59e7 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:21.300631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35a7ac-c747-4313-b28e-6731b7a07474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:21.301249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35a7ac-c747-4313-b28e-6731b7a07474 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:21.301519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35a7ac-c747-4313-b28e-6731b7a07474 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:21.301788 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35a7ac-c747-4313-b28e-6731b7a07474 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:21.306602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013680e9-cfa2-49a2-9812-e3b8f16ea5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:21.307220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013680e9-cfa2-49a2-9812-e3b8f16ea5f5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:21.307490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013680e9-cfa2-49a2-9812-e3b8f16ea5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:21.307752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013680e9-cfa2-49a2-9812-e3b8f16ea5f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:21.308914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35a7ac-c747-4313-b28e-6731b7a07474 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:21.310367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35a7ac-c747-4313-b28e-6731b7a07474 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:21.319518 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013680e9-cfa2-49a2-9812-e3b8f16ea5f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-fd35a7ac-c747-4313-b28e-6731b7a07474 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:21.322155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:21.323046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:21.324098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:21.324098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:21.324098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:21.324098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:21.324098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:21.324098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:21.324098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:21.324098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:21.324098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:24:21.324098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-013680e9-cfa2-49a2-9812-e3b8f16ea5f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:21.325140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:21.326015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 285/562] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:21.329106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013680e9-cfa2-49a2-9812-e3b8f16ea5f5 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:21.332106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 278/563] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 16 05:24:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:21.332106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166c195-e352-4032-a8af-b271867cfbbd None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:21.332106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166c195-e352-4032-a8af-b271867cfbbd None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:21.332106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166c195-e352-4032-a8af-b271867cfbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:21.348975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f166c195-e352-4032-a8af-b271867cfbbd None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:21.350188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f166c195-e352-4032-a8af-b271867cfbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:21.420921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 279/564] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:21.423041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166c195-e352-4032-a8af-b271867cfbbd None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:21.430826 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:21.438098 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013680e9-cfa2-49a2-9812-e3b8f16ea5f5 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:21.472794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03c9f3-5bc2-42bc-9be7-5a3194294889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:21.473345 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03c9f3-5bc2-42bc-9be7-5a3194294889 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:21.473621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03c9f3-5bc2-42bc-9be7-5a3194294889 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:21.473925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03c9f3-5bc2-42bc-9be7-5a3194294889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:21.485111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03c9f3-5bc2-42bc-9be7-5a3194294889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:21.486586 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03c9f3-5bc2-42bc-9be7-5a3194294889 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-ed03c9f3-5bc2-42bc-9be7-5a3194294889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:21.501512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:21.502494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:21.503378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:21.503378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:21.503378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:21.506473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 286/565] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:21.507802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03c9f3-5bc2-42bc-9be7-5a3194294889 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:21.520871 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:21.530010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f166c195-e352-4032-a8af-b271867cfbbd None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:21.565524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae6be4-c70b-41ac-a6e4-b7494689cfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:21.566169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae6be4-c70b-41ac-a6e4-b7494689cfe7 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:21.566451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae6be4-c70b-41ac-a6e4-b7494689cfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:21.566701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae6be4-c70b-41ac-a6e4-b7494689cfe7 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:21.567430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae6be4-c70b-41ac-a6e4-b7494689cfe7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:21.567734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae6be4-c70b-41ac-a6e4-b7494689cfe7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:21.569145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae6be4-c70b-41ac-a6e4-b7494689cfe7 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:21.587242 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 280/566] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 16 05:24:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:21.588054 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae6be4-c70b-41ac-a6e4-b7494689cfe7 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:21.619110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03c9f3-5bc2-42bc-9be7-5a3194294889 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:21.654374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b9d4d-fef4-435a-a662-cb4c542bc8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:21.654979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b9d4d-fef4-435a-a662-cb4c542bc8cd None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:21.655259 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b9d4d-fef4-435a-a662-cb4c542bc8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:21.655533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b9d4d-fef4-435a-a662-cb4c542bc8cd None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:21.657028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b9d4d-fef4-435a-a662-cb4c542bc8cd None admin] RBAC: Adding query filter params (name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:21.657429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b9d4d-fef4-435a-a662-cb4c542bc8cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:21.658844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b9d4d-fef4-435a-a662-cb4c542bc8cd None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:21.666966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 287/567] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:21.666966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b9d4d-fef4-435a-a662-cb4c542bc8cd None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:21.682711 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:21.698181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae6be4-c70b-41ac-a6e4-b7494689cfe7 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:21.731792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ba97f-c1d1-4fc6-a74a-dfadf33aa18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:21.732342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ba97f-c1d1-4fc6-a74a-dfadf33aa18b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:21.732616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ba97f-c1d1-4fc6-a74a-dfadf33aa18b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:21.732882 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ba97f-c1d1-4fc6-a74a-dfadf33aa18b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:21.743480 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ba97f-c1d1-4fc6-a74a-dfadf33aa18b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-dd4ba97f-c1d1-4fc6-a74a-dfadf33aa18b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:21.748331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:21.749270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 281/568] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:21.750930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ba97f-c1d1-4fc6-a74a-dfadf33aa18b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:21.764129 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:21.773723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b9d4d-fef4-435a-a662-cb4c542bc8cd None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:21.809684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed697e-195b-462a-99af-1942f589f6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:21.810311 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed697e-195b-462a-99af-1942f589f6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:21.810583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed697e-195b-462a-99af-1942f589f6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:21.810856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed697e-195b-462a-99af-1942f589f6b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:21.811485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed697e-195b-462a-99af-1942f589f6b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:21.811783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed697e-195b-462a-99af-1942f589f6b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:21.814110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed697e-195b-462a-99af-1942f589f6b2 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:21.825587 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 288/569] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:21.826407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed697e-195b-462a-99af-1942f589f6b2 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:21.853594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ba97f-c1d1-4fc6-a74a-dfadf33aa18b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:21.887492 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50aeed-7931-4758-816b-def96564e105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:21.888106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50aeed-7931-4758-816b-def96564e105 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:21.888377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50aeed-7931-4758-816b-def96564e105 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:21.888643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50aeed-7931-4758-816b-def96564e105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:21.891278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:21.899958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50aeed-7931-4758-816b-def96564e105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:21.901394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50aeed-7931-4758-816b-def96564e105 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-4d50aeed-7931-4758-816b-def96564e105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:21.916248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:21.917240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:21.918138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:21.918138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:21.918138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:21.923025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 282/570] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:21.923860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50aeed-7931-4758-816b-def96564e105 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:21.926678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed697e-195b-462a-99af-1942f589f6b2 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:21.961061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c2278-8fee-46e8-a26e-40b4039674fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:21.961720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c2278-8fee-46e8-a26e-40b4039674fe None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:21.961964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c2278-8fee-46e8-a26e-40b4039674fe None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:21.962227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c2278-8fee-46e8-a26e-40b4039674fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:21.971921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c2278-8fee-46e8-a26e-40b4039674fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:21.973355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c2278-8fee-46e8-a26e-40b4039674fe None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:21.983270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-510c2278-8fee-46e8-a26e-40b4039674fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:21.988016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:21.989101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:21.990029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:21.990029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:21.990029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:21.990029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 289/571] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:21.990299 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c2278-8fee-46e8-a26e-40b4039674fe None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:22.025438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50aeed-7931-4758-816b-def96564e105 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:22.057641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c937c3-095f-472c-a19b-e4b09954c3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:22.058279 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c937c3-095f-472c-a19b-e4b09954c3cc None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:22.058547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c937c3-095f-472c-a19b-e4b09954c3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:22.058807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c937c3-095f-472c-a19b-e4b09954c3cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:22.069375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c937c3-095f-472c-a19b-e4b09954c3cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-65c937c3-095f-472c-a19b-e4b09954c3cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:22.074569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:22.075684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 283/572] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:22.076529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c937c3-095f-472c-a19b-e4b09954c3cc None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:22.095393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c2278-8fee-46e8-a26e-40b4039674fe None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:22.129912 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:22.144230 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe2d43-04d5-43de-8c26-d18ba1dd79d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:22.148925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe2d43-04d5-43de-8c26-d18ba1dd79d9 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:22.148925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe2d43-04d5-43de-8c26-d18ba1dd79d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:22.148925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe2d43-04d5-43de-8c26-d18ba1dd79d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:22.148925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe2d43-04d5-43de-8c26-d18ba1dd79d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:22.148925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe2d43-04d5-43de-8c26-d18ba1dd79d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:22.148925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe2d43-04d5-43de-8c26-d18ba1dd79d9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:22.155867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 290/573] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:22.156903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe2d43-04d5-43de-8c26-d18ba1dd79d9 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:22.175598 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c937c3-095f-472c-a19b-e4b09954c3cc None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:22.195123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:22.208429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d1abc-2886-467e-9632-2ad5808c0362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:22.209045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d1abc-2886-467e-9632-2ad5808c0362 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:22.209315 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d1abc-2886-467e-9632-2ad5808c0362 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:22.209578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d1abc-2886-467e-9632-2ad5808c0362 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:22.215978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d1abc-2886-467e-9632-2ad5808c0362 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:22.217387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d1abc-2886-467e-9632-2ad5808c0362 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-4b6d1abc-2886-467e-9632-2ad5808c0362 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:22.229244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:22.230374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:22.236766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:22.236766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:22.236766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:22.236766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:22.236766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:22.236766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:22.236766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:22.236766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:22.236766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 284/574] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:22.236766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d1abc-2886-467e-9632-2ad5808c0362 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:22.256508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe2d43-04d5-43de-8c26-d18ba1dd79d9 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:22.289557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae629e-7f50-46c9-aa5a-b52db3eb8380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:22.290184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae629e-7f50-46c9-aa5a-b52db3eb8380 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:22.290457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae629e-7f50-46c9-aa5a-b52db3eb8380 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:22.290807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae629e-7f50-46c9-aa5a-b52db3eb8380 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:22.291352 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae629e-7f50-46c9-aa5a-b52db3eb8380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:22.291653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae629e-7f50-46c9-aa5a-b52db3eb8380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:22.293008 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae629e-7f50-46c9-aa5a-b52db3eb8380 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:22.305562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 291/575] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:22.306407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae629e-7f50-46c9-aa5a-b52db3eb8380 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:22.332996 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d1abc-2886-467e-9632-2ad5808c0362 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:22.348161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:22.366916 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb46fc1-6e6d-476e-ac09-44d3fc9a0cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:22.367520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb46fc1-6e6d-476e-ac09-44d3fc9a0cea None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:22.367793 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb46fc1-6e6d-476e-ac09-44d3fc9a0cea None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:22.368148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb46fc1-6e6d-476e-ac09-44d3fc9a0cea None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:22.368788 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb46fc1-6e6d-476e-ac09-44d3fc9a0cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:22.369079 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb46fc1-6e6d-476e-ac09-44d3fc9a0cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:22.370477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb46fc1-6e6d-476e-ac09-44d3fc9a0cea None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:22.387441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 285/576] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:22.387441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb46fc1-6e6d-476e-ac09-44d3fc9a0cea None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:22.403393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae629e-7f50-46c9-aa5a-b52db3eb8380 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:22.427314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:22.437656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6294f92-e3dd-4dff-af18-0e9c74d0e80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:22.438400 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6294f92-e3dd-4dff-af18-0e9c74d0e80e None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:22.438668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6294f92-e3dd-4dff-af18-0e9c74d0e80e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:22.438958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6294f92-e3dd-4dff-af18-0e9c74d0e80e None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:22.439705 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6294f92-e3dd-4dff-af18-0e9c74d0e80e None admin] RBAC: Adding query filter params (name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:22.440000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6294f92-e3dd-4dff-af18-0e9c74d0e80e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:22.441335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6294f92-e3dd-4dff-af18-0e9c74d0e80e None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:22.448932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 292/577] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:22.449720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6294f92-e3dd-4dff-af18-0e9c74d0e80e None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:22.483157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb46fc1-6e6d-476e-ac09-44d3fc9a0cea None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:22.517745 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d26c7-aa90-4f46-adb9-8d84339bf306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:22.518319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d26c7-aa90-4f46-adb9-8d84339bf306 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:22.518588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d26c7-aa90-4f46-adb9-8d84339bf306 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:22.518845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d26c7-aa90-4f46-adb9-8d84339bf306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:22.528466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d26c7-aa90-4f46-adb9-8d84339bf306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:22.529907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d26c7-aa90-4f46-adb9-8d84339bf306 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-392d26c7-aa90-4f46-adb9-8d84339bf306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:22.545106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:22.546172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:22.547131 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:22.547131 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:22.547131 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:22.547131 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 286/578] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:22.547356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6294f92-e3dd-4dff-af18-0e9c74d0e80e None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:22.547429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d26c7-aa90-4f46-adb9-8d84339bf306 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:22.552157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:22.584238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2689a6c-0f19-486f-9b05-7b26ceed9466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:22.584836 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2689a6c-0f19-486f-9b05-7b26ceed9466 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:22.585112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2689a6c-0f19-486f-9b05-7b26ceed9466 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:22.585379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2689a6c-0f19-486f-9b05-7b26ceed9466 None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:22.586515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2689a6c-0f19-486f-9b05-7b26ceed9466 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:22.586632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2689a6c-0f19-486f-9b05-7b26ceed9466 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:22.588324 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2689a6c-0f19-486f-9b05-7b26ceed9466 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:22.605693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 293/579] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:22.606526 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2689a6c-0f19-486f-9b05-7b26ceed9466 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:22.646595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:22.655750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d26c7-aa90-4f46-adb9-8d84339bf306 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:22.690236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d228820-807f-47ef-b26a-c77928645fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:22.690861 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d228820-807f-47ef-b26a-c77928645fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:22.691140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d228820-807f-47ef-b26a-c77928645fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:22.691414 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d228820-807f-47ef-b26a-c77928645fc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:22.698036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d228820-807f-47ef-b26a-c77928645fc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:22.699459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d228820-807f-47ef-b26a-c77928645fc0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:22.702429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2689a6c-0f19-486f-9b05-7b26ceed9466 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-5d228820-807f-47ef-b26a-c77928645fc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:22.711137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:22.712042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:22.712964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:22.712964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:22.712964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:22.712964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:22.712964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:22.712964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:22.712964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:22.712964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:22.712964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 287/580] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:22.712964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d228820-807f-47ef-b26a-c77928645fc0 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:22.739991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3ae8c-d202-4cc7-8120-f894e1190258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:22.740614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3ae8c-d202-4cc7-8120-f894e1190258 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:22.740889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3ae8c-d202-4cc7-8120-f894e1190258 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:22.741155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3ae8c-d202-4cc7-8120-f894e1190258 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:22.748064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3ae8c-d202-4cc7-8120-f894e1190258 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:22.749588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3ae8c-d202-4cc7-8120-f894e1190258 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-80d3ae8c-d202-4cc7-8120-f894e1190258 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:22.763420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:22.764359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:22.766426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:22.766426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:22.766426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:22.766426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:22.766426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:22.766426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:22.766426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:22.766426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:22.766426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 16 05:24:22.766426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:22.766426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 294/581] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:22.766426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3ae8c-d202-4cc7-8120-f894e1190258 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:22.790641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:22.813720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d228820-807f-47ef-b26a-c77928645fc0 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:22.847155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc81366-58fe-4c26-ba38-ec40230b117d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:22.847766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc81366-58fe-4c26-ba38-ec40230b117d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:22.848036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc81366-58fe-4c26-ba38-ec40230b117d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:22.848297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc81366-58fe-4c26-ba38-ec40230b117d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:22.857992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc81366-58fe-4c26-ba38-ec40230b117d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:22.859467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc81366-58fe-4c26-ba38-ec40230b117d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:22.863959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3ae8c-d202-4cc7-8120-f894e1190258 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-1bc81366-58fe-4c26-ba38-ec40230b117d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:22.874327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:22.875491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:22.876383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:22.876383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:22.876383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:22.876383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 288/582] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:22.876383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc81366-58fe-4c26-ba38-ec40230b117d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:22.879978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:22.901035 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bea41-036a-44ee-9f18-268602f8fec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:22.901627 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bea41-036a-44ee-9f18-268602f8fec9 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:22.901934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bea41-036a-44ee-9f18-268602f8fec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:22.902200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bea41-036a-44ee-9f18-268602f8fec9 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:22.902835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bea41-036a-44ee-9f18-268602f8fec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:22.903127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bea41-036a-44ee-9f18-268602f8fec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:22.904478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bea41-036a-44ee-9f18-268602f8fec9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:22.918146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 295/583] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:22.919196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bea41-036a-44ee-9f18-268602f8fec9 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:22.976952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc81366-58fe-4c26-ba38-ec40230b117d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:23.003025 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:23.011938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b14571-43a1-48aa-b3d3-c307241f5262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:23.012530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b14571-43a1-48aa-b3d3-c307241f5262 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:23.012797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b14571-43a1-48aa-b3d3-c307241f5262 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:23.014900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b14571-43a1-48aa-b3d3-c307241f5262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:23.014900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b14571-43a1-48aa-b3d3-c307241f5262 None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=b260fed0d99a4d3fa8f9c2ad79223219) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:23.014900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b14571-43a1-48aa-b3d3-c307241f5262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:23.015604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b14571-43a1-48aa-b3d3-c307241f5262 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:23.018730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bea41-036a-44ee-9f18-268602f8fec9 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:23.021077 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 289/584] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:22 2022] GET /identity/v3/role_assignments?user.id=b260fed0d99a4d3fa8f9c2ad79223219&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:23.021851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b14571-43a1-48aa-b3d3-c307241f5262 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:23.055813 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c29a1-4760-48f5-889d-09c397326335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:23.056351 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c29a1-4760-48f5-889d-09c397326335 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:23.056629 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c29a1-4760-48f5-889d-09c397326335 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:23.056898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c29a1-4760-48f5-889d-09c397326335 None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:23.057605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c29a1-4760-48f5-889d-09c397326335 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:23.057941 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c29a1-4760-48f5-889d-09c397326335 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:23.059397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c29a1-4760-48f5-889d-09c397326335 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:23.072495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 296/585] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:23.073375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c29a1-4760-48f5-889d-09c397326335 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:23.107230 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:23.146608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b14571-43a1-48aa-b3d3-c307241f5262 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:23.187223 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de27ee7-7500-4bde-80af-e02ae4d6284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:23.187899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de27ee7-7500-4bde-80af-e02ae4d6284b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:23.188175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de27ee7-7500-4bde-80af-e02ae4d6284b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:23.188442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de27ee7-7500-4bde-80af-e02ae4d6284b None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:23.189175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de27ee7-7500-4bde-80af-e02ae4d6284b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:23.189477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de27ee7-7500-4bde-80af-e02ae4d6284b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:23.190943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de27ee7-7500-4bde-80af-e02ae4d6284b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:23.194318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c29a1-4760-48f5-889d-09c397326335 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:23.214691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 290/586] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Sep 16 05:24:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:23.214889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de27ee7-7500-4bde-80af-e02ae4d6284b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:23.234689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea6d8a-d2ff-4fd7-a365-b2db1941466d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:23.235403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea6d8a-d2ff-4fd7-a365-b2db1941466d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:23.235674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea6d8a-d2ff-4fd7-a365-b2db1941466d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:23.235934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea6d8a-d2ff-4fd7-a365-b2db1941466d None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:23.236569 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea6d8a-d2ff-4fd7-a365-b2db1941466d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:23.236863 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea6d8a-d2ff-4fd7-a365-b2db1941466d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:23.238369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea6d8a-d2ff-4fd7-a365-b2db1941466d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:23.255928 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 297/587] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:23.256643 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea6d8a-d2ff-4fd7-a365-b2db1941466d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:23.260237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:23.321510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de27ee7-7500-4bde-80af-e02ae4d6284b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:23.350253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:23.358560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea6d8a-d2ff-4fd7-a365-b2db1941466d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:23.361948 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef5f3e-6ea9-4bc4-9772-ed4ceeac49e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:23.361948 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef5f3e-6ea9-4bc4-9772-ed4ceeac49e2 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:23.361948 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef5f3e-6ea9-4bc4-9772-ed4ceeac49e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:23.361948 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef5f3e-6ea9-4bc4-9772-ed4ceeac49e2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:23.368027 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef5f3e-6ea9-4bc4-9772-ed4ceeac49e2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:23.369458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef5f3e-6ea9-4bc4-9772-ed4ceeac49e2 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-9bef5f3e-6ea9-4bc4-9772-ed4ceeac49e2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:23.381309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:23.382286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:23.383278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:23.383278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:23.383278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:23.383278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:23.383278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:23.383278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:23.383278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:23.383278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:23.383278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 05:24:23.383278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:23.383278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 291/588] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:23.383278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef5f3e-6ea9-4bc4-9772-ed4ceeac49e2 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:23.407642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1684d-82c0-4c27-bf60-1117c62344b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:23.408811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1684d-82c0-4c27-bf60-1117c62344b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:23.409234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1684d-82c0-4c27-bf60-1117c62344b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:23.409784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1684d-82c0-4c27-bf60-1117c62344b7 None admin] PATH_INFO: `/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/7db75c20e9744feb8b5dc78c554f4977/roles/ee551384189f433bba7e06f47a794cf0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:23.429698 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1684d-82c0-4c27-bf60-1117c62344b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=29b2e27932984de79ee8dfbe95312ac2, user_id=7db75c20e9744feb8b5dc78c554f4977, role_id=ee551384189f433bba7e06f47a794cf0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:23.431644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1684d-82c0-4c27-bf60-1117c62344b7 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:23.502060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 298/589] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:24:23 2022] PUT /identity/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/7db75c20e9744feb8b5dc78c554f4977/roles/ee551384189f433bba7e06f47a794cf0 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:24:23.503256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1684d-82c0-4c27-bf60-1117c62344b7 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:23.507600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef5f3e-6ea9-4bc4-9772-ed4ceeac49e2 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:23.549517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e18f9-23da-461d-bff5-844071ad993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:23.550247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e18f9-23da-461d-bff5-844071ad993f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:23.550531 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e18f9-23da-461d-bff5-844071ad993f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:23.550936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e18f9-23da-461d-bff5-844071ad993f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:23.566891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e18f9-23da-461d-bff5-844071ad993f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:23.568897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401e18f9-23da-461d-bff5-844071ad993f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-401e18f9-23da-461d-bff5-844071ad993f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:23.589622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:23.590663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:23.591561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:23.591561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:23.591561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:23.591561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 292/590] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:23.591561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e18f9-23da-461d-bff5-844071ad993f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:23.594143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:23.644500 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:23.652465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1684d-82c0-4c27-bf60-1117c62344b7 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:23.692052 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d88e69-4fde-4c4d-9556-2bf25bcb3272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:23.693051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d88e69-4fde-4c4d-9556-2bf25bcb3272 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:23.693567 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d88e69-4fde-4c4d-9556-2bf25bcb3272 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:23.694190 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d88e69-4fde-4c4d-9556-2bf25bcb3272 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:23.702115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d88e69-4fde-4c4d-9556-2bf25bcb3272 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:23.704393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d88e69-4fde-4c4d-9556-2bf25bcb3272 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:23.712502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401e18f9-23da-461d-bff5-844071ad993f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-37d88e69-4fde-4c4d-9556-2bf25bcb3272 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:23.717227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:23.722719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:23.723667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:23.723667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:23.723667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:23.723667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:23.723667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:23.723667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:23.723667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:23.723667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:23.723667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 05:24:23.723667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:23.723667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 299/591] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:23.723667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d88e69-4fde-4c4d-9556-2bf25bcb3272 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:23.755404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb27e0-da33-4d73-895d-4c555a2aadce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:23.756389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb27e0-da33-4d73-895d-4c555a2aadce None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:23.756784 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb27e0-da33-4d73-895d-4c555a2aadce None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:23.757152 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb27e0-da33-4d73-895d-4c555a2aadce None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:23.758218 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb27e0-da33-4d73-895d-4c555a2aadce None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:23.758621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb27e0-da33-4d73-895d-4c555a2aadce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:23.760622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb27e0-da33-4d73-895d-4c555a2aadce None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:23.788553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 293/592] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Sep 16 05:24:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:23.793082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb27e0-da33-4d73-895d-4c555a2aadce None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:23.855575 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d88e69-4fde-4c4d-9556-2bf25bcb3272 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:23.872240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:23.895375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f35f3-41d0-4999-a53c-5ec27eef46b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:23.896440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f35f3-41d0-4999-a53c-5ec27eef46b9 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:23.896910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f35f3-41d0-4999-a53c-5ec27eef46b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:23.897423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f35f3-41d0-4999-a53c-5ec27eef46b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:23.898405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f35f3-41d0-4999-a53c-5ec27eef46b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:23.898916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f35f3-41d0-4999-a53c-5ec27eef46b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:23.900565 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f35f3-41d0-4999-a53c-5ec27eef46b9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:23.915796 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:23.922806 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb27e0-da33-4d73-895d-4c555a2aadce None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:23.928446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 300/593] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:23.935448 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f35f3-41d0-4999-a53c-5ec27eef46b9 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:23.971760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338b856-2839-433b-b7d7-e07d49f69bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:23.972371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338b856-2839-433b-b7d7-e07d49f69bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:23.972647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338b856-2839-433b-b7d7-e07d49f69bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:23.972915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338b856-2839-433b-b7d7-e07d49f69bb4 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:23.973639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338b856-2839-433b-b7d7-e07d49f69bb4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:23.973969 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338b856-2839-433b-b7d7-e07d49f69bb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:23.975438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338b856-2839-433b-b7d7-e07d49f69bb4 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:24.001938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 294/594] 10.222.0.49 () {56 vars in 1139 bytes} [Fri Sep 16 05:24:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:24.003061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338b856-2839-433b-b7d7-e07d49f69bb4 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:24.047357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f35f3-41d0-4999-a53c-5ec27eef46b9 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:24.096070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15b977-2f2d-4f49-9356-a43739f6002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:24.097023 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15b977-2f2d-4f49-9356-a43739f6002b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:24.097501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15b977-2f2d-4f49-9356-a43739f6002b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:24.098007 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15b977-2f2d-4f49-9356-a43739f6002b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:24.108918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15b977-2f2d-4f49-9356-a43739f6002b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:24.110659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15b977-2f2d-4f49-9356-a43739f6002b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-bb15b977-2f2d-4f49-9356-a43739f6002b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:24.126287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:24.127277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:24.128152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:24.128152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:24.128152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:24.133934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 301/595] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:24.134971 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15b977-2f2d-4f49-9356-a43739f6002b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:24.141939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338b856-2839-433b-b7d7-e07d49f69bb4 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:24.165947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:24.192816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fab5e3-06dc-4280-bedd-41b0daffd0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:24.194009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fab5e3-06dc-4280-bedd-41b0daffd0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:24.194745 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fab5e3-06dc-4280-bedd-41b0daffd0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:24.195214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fab5e3-06dc-4280-bedd-41b0daffd0c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:24.205314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fab5e3-06dc-4280-bedd-41b0daffd0c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:24.207297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fab5e3-06dc-4280-bedd-41b0daffd0c8 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-25fab5e3-06dc-4280-bedd-41b0daffd0c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:24.222371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:24.223286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:24.224166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:24.224166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:24.224166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:24.224166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:24.224166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:24.224166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:24.224166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:24.224166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:24.227122 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 295/596] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:24.228100 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fab5e3-06dc-4280-bedd-41b0daffd0c8 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:24.249322 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:24.254282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15b977-2f2d-4f49-9356-a43739f6002b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:24.300768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae4fde-1321-495e-93be-64da8d3c8d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:24.301419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae4fde-1321-495e-93be-64da8d3c8d3d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:24.301705 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae4fde-1321-495e-93be-64da8d3c8d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:24.302005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae4fde-1321-495e-93be-64da8d3c8d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:24.319937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae4fde-1321-495e-93be-64da8d3c8d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:24.320822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae4fde-1321-495e-93be-64da8d3c8d3d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-35ae4fde-1321-495e-93be-64da8d3c8d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:24.337517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:24.338724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:24.339707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:24.339707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:24.339707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:24.343846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 302/597] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:24.344962 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae4fde-1321-495e-93be-64da8d3c8d3d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:24.358279 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fab5e3-06dc-4280-bedd-41b0daffd0c8 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:24.400812 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d06bd1-3e30-48ca-82d2-133211e3fc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:24.401783 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d06bd1-3e30-48ca-82d2-133211e3fc60 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:24.402319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d06bd1-3e30-48ca-82d2-133211e3fc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:24.402784 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d06bd1-3e30-48ca-82d2-133211e3fc60 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:24.403718 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d06bd1-3e30-48ca-82d2-133211e3fc60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:24.404228 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d06bd1-3e30-48ca-82d2-133211e3fc60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:24.406472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d06bd1-3e30-48ca-82d2-133211e3fc60 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:24.420865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 296/598] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:24.424686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d06bd1-3e30-48ca-82d2-133211e3fc60 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:24.433698 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:24.460892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae4fde-1321-495e-93be-64da8d3c8d3d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:24.498153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171cd71-c3b9-446e-94ea-d1a0d8cee3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:24.498757 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171cd71-c3b9-446e-94ea-d1a0d8cee3eb None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:24.499034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171cd71-c3b9-446e-94ea-d1a0d8cee3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:24.499304 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3171cd71-c3b9-446e-94ea-d1a0d8cee3eb None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:24.500097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3171cd71-c3b9-446e-94ea-d1a0d8cee3eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:24.500759 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3171cd71-c3b9-446e-94ea-d1a0d8cee3eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:24.502006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3171cd71-c3b9-446e-94ea-d1a0d8cee3eb None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:24.515838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 303/599] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:24.516473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171cd71-c3b9-446e-94ea-d1a0d8cee3eb None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:24.529624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:24.563795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d06bd1-3e30-48ca-82d2-133211e3fc60 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:24.611763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7ea67-7810-4518-b17e-568b34a8cae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:24.612727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7ea67-7810-4518-b17e-568b34a8cae7 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:24.613071 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7ea67-7810-4518-b17e-568b34a8cae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:24.613412 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7ea67-7810-4518-b17e-568b34a8cae7 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:24.614590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7ea67-7810-4518-b17e-568b34a8cae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:24.614960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7ea67-7810-4518-b17e-568b34a8cae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:24.616407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7ea67-7810-4518-b17e-568b34a8cae7 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:24.634422 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3171cd71-c3b9-446e-94ea-d1a0d8cee3eb None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:24.635265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 297/600] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:24.636272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7ea67-7810-4518-b17e-568b34a8cae7 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:24.683015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb75f0-15a9-4868-aa7b-037a189869ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:24.684109 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb75f0-15a9-4868-aa7b-037a189869ac None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:24.684463 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb75f0-15a9-4868-aa7b-037a189869ac None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:24.684805 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb75f0-15a9-4868-aa7b-037a189869ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:24.694626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb75f0-15a9-4868-aa7b-037a189869ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:24.696077 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb75f0-15a9-4868-aa7b-037a189869ac None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:24.698966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-16fb75f0-15a9-4868-aa7b-037a189869ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:24.709183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:24.710146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:24.711062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:24.711062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:24.711062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:24.711062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:24.711062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:24.711062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:24.711062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:24.711062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:24.711062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 304/601] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:24.711565 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb75f0-15a9-4868-aa7b-037a189869ac None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:24.768892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7ea67-7810-4518-b17e-568b34a8cae7 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:24.816020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a082401-ba60-4d2e-8380-065b50e8d729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:24.817171 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a082401-ba60-4d2e-8380-065b50e8d729 None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:24.817615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a082401-ba60-4d2e-8380-065b50e8d729 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:24.818252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a082401-ba60-4d2e-8380-065b50e8d729 None admin] PATH_INFO: `/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/30d96ec6235843c5b67d87a3d0ff0f8f/roles/ee551384189f433bba7e06f47a794cf0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:24.821654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb75f0-15a9-4868-aa7b-037a189869ac None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:24.838176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:24.842383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a082401-ba60-4d2e-8380-065b50e8d729 None admin] RBAC: Authorizing `identity:create_grant(project_id=29b2e27932984de79ee8dfbe95312ac2, user_id=30d96ec6235843c5b67d87a3d0ff0f8f, role_id=ee551384189f433bba7e06f47a794cf0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:24.845399 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a082401-ba60-4d2e-8380-065b50e8d729 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:24.859169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b1f34-0401-416e-abdb-33a94399db6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:24.863968 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b1f34-0401-416e-abdb-33a94399db6f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:24.863968 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b1f34-0401-416e-abdb-33a94399db6f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:24.863968 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b1f34-0401-416e-abdb-33a94399db6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:24.868141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b1f34-0401-416e-abdb-33a94399db6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:24.869658 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b1f34-0401-416e-abdb-33a94399db6f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-1a3b1f34-0401-416e-abdb-33a94399db6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:24.881857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:24.882760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:24.883695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:24.883695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:24.883695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:24.883695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:24.883695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:24.883695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:24.883695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:24.883695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:24.883695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 305/602] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:24.884384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b1f34-0401-416e-abdb-33a94399db6f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:24.933247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 298/603] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:24:24 2022] PUT /identity/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/30d96ec6235843c5b67d87a3d0ff0f8f/roles/ee551384189f433bba7e06f47a794cf0 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:24:24.934440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a082401-ba60-4d2e-8380-065b50e8d729 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:24.978482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:25.091937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b1f34-0401-416e-abdb-33a94399db6f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:25.124011 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a082401-ba60-4d2e-8380-065b50e8d729 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:25.145748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93250fa2-d2aa-469b-8405-5557e4a1a1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:25.146722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93250fa2-d2aa-469b-8405-5557e4a1a1fe None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:25.147192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93250fa2-d2aa-469b-8405-5557e4a1a1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:25.147699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93250fa2-d2aa-469b-8405-5557e4a1a1fe None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:25.149355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93250fa2-d2aa-469b-8405-5557e4a1a1fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:25.149993 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93250fa2-d2aa-469b-8405-5557e4a1a1fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:25.151756 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93250fa2-d2aa-469b-8405-5557e4a1a1fe None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:25.169862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 306/604] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:25.170080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69922870-1367-4773-a652-ba332e4d81f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:25.170829 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69922870-1367-4773-a652-ba332e4d81f9 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:25.170932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69922870-1367-4773-a652-ba332e4d81f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:25.172295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69922870-1367-4773-a652-ba332e4d81f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:25.172628 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69922870-1367-4773-a652-ba332e4d81f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:25.173700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93250fa2-d2aa-469b-8405-5557e4a1a1fe None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:25.177885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69922870-1367-4773-a652-ba332e4d81f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:25.179641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69922870-1367-4773-a652-ba332e4d81f9 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:25.198347 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 299/605] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:25.200527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69922870-1367-4773-a652-ba332e4d81f9 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:25.285062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:25.298804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93250fa2-d2aa-469b-8405-5557e4a1a1fe None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:25.323181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:25.339898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0aa88a-5e8b-4e57-8199-a13634d8fd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:25.345844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0aa88a-5e8b-4e57-8199-a13634d8fd0a None admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:25.345844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0aa88a-5e8b-4e57-8199-a13634d8fd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:25.345844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0aa88a-5e8b-4e57-8199-a13634d8fd0a None admin] PATH_INFO: `/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/ff50e50587254bac912fb342a7e09891/roles/ee551384189f433bba7e06f47a794cf0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:25.363199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0aa88a-5e8b-4e57-8199-a13634d8fd0a None admin] RBAC: Authorizing `identity:create_grant(project_id=29b2e27932984de79ee8dfbe95312ac2, user_id=ff50e50587254bac912fb342a7e09891, role_id=ee551384189f433bba7e06f47a794cf0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:25.365140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0aa88a-5e8b-4e57-8199-a13634d8fd0a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:25.382875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69922870-1367-4773-a652-ba332e4d81f9 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:25.426458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82cedb-0e71-4d88-b94e-ed337c7c1a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:25.427112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82cedb-0e71-4d88-b94e-ed337c7c1a02 None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:25.427387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82cedb-0e71-4d88-b94e-ed337c7c1a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:25.427657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82cedb-0e71-4d88-b94e-ed337c7c1a02 None admin] PATH_INFO: `/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/3e923be9aeed43728690d24d85dbe062/roles/ee551384189f433bba7e06f47a794cf0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:25.443728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82cedb-0e71-4d88-b94e-ed337c7c1a02 None admin] RBAC: Authorizing `identity:create_grant(project_id=29b2e27932984de79ee8dfbe95312ac2, user_id=3e923be9aeed43728690d24d85dbe062, role_id=ee551384189f433bba7e06f47a794cf0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:25.445181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82cedb-0e71-4d88-b94e-ed337c7c1a02 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:25.474102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 307/606] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:24:25 2022] PUT /identity/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/ff50e50587254bac912fb342a7e09891/roles/ee551384189f433bba7e06f47a794cf0 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:24:25.475960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0f941-b0d8-4d8f-a05c-626c71758091 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:25.476375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0f941-b0d8-4d8f-a05c-626c71758091 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:25.476770 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0f941-b0d8-4d8f-a05c-626c71758091 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:25.478901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 308/607] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:25.491423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa5fe6-7193-447c-b25f-acbe2a4fe042 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:25.491934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa5fe6-7193-447c-b25f-acbe2a4fe042 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:25.492343 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa5fe6-7193-447c-b25f-acbe2a4fe042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:25.513126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-fcfa5fe6-7193-447c-b25f-acbe2a4fe042 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:25.514684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-fcfa5fe6-7193-447c-b25f-acbe2a4fe042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:25.553879 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 300/608] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:24:25 2022] PUT /identity/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/3e923be9aeed43728690d24d85dbe062/roles/ee551384189f433bba7e06f47a794cf0 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:24:25.624988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 309/609] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:25.828730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b9110-8cc5-4ce9-8d9b-d1dd0c2f9064 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:25.828918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b9110-8cc5-4ce9-8d9b-d1dd0c2f9064 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:25.829270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b9110-8cc5-4ce9-8d9b-d1dd0c2f9064 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:25.830872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 301/610] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:25.836433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e86b6-c249-428f-8413-f55e7db6c335 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:25.836721 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e86b6-c249-428f-8413-f55e7db6c335 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:25.836993 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e86b6-c249-428f-8413-f55e7db6c335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:25.856007 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-677e86b6-c249-428f-8413-f55e7db6c335 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:25.857200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-677e86b6-c249-428f-8413-f55e7db6c335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:25.925481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b9110-8cc5-4ce9-8d9b-d1dd0c2f9064 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:25.955145 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 310/611] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:26.071125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:26.079319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b9110-8cc5-4ce9-8d9b-d1dd0c2f9064 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:26.123808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b3cb9-a553-4c32-ab98-0c1ae7802f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:26.125071 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b3cb9-a553-4c32-ab98-0c1ae7802f46 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:26.125356 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b3cb9-a553-4c32-ab98-0c1ae7802f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:26.125622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78b3cb9-a553-4c32-ab98-0c1ae7802f46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:26.142684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78b3cb9-a553-4c32-ab98-0c1ae7802f46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-a78b3cb9-a553-4c32-ab98-0c1ae7802f46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:26.147681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:26.148588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:26.149417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:26.149417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:26.149417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:26.149417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:26.149417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:26.149417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:26.149417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:26.149417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:26.149417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:26.149417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:26.149417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:26.149417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:26.151404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 302/612] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:26.158210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e86b6-c249-428f-8413-f55e7db6c335 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:26.196497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:26.293127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e86b6-c249-428f-8413-f55e7db6c335 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:26.343199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41d10c-a9ee-4388-97eb-381c06d5551a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:26.343744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41d10c-a9ee-4388-97eb-381c06d5551a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:26.344077 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41d10c-a9ee-4388-97eb-381c06d5551a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:26.344351 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41d10c-a9ee-4388-97eb-381c06d5551a None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:26.345016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41d10c-a9ee-4388-97eb-381c06d5551a None admin] RBAC: Adding query filter params (name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:26.345375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41d10c-a9ee-4388-97eb-381c06d5551a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:26.346771 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41d10c-a9ee-4388-97eb-381c06d5551a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:26.358554 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 311/613] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:26.363941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b3cb9-a553-4c32-ab98-0c1ae7802f46 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:26.446390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41d10c-a9ee-4388-97eb-381c06d5551a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:26.504756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78b3cb9-a553-4c32-ab98-0c1ae7802f46 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:26.545935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383a520-1a2f-4ec0-bab4-54ebb7a9afc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:26.547111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383a520-1a2f-4ec0-bab4-54ebb7a9afc1 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:26.547389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383a520-1a2f-4ec0-bab4-54ebb7a9afc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:26.547775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383a520-1a2f-4ec0-bab4-54ebb7a9afc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:26.570464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383a520-1a2f-4ec0-bab4-54ebb7a9afc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:26.571927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383a520-1a2f-4ec0-bab4-54ebb7a9afc1 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-8383a520-1a2f-4ec0-bab4-54ebb7a9afc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:26.605398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:26.606393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:26.607261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:26.607261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:26.607261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:26.607261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 303/614] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:26.607488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383a520-1a2f-4ec0-bab4-54ebb7a9afc1 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:26.613470 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:26.623191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:26.632018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41d10c-a9ee-4388-97eb-381c06d5551a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:26.694040 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e19ce5-b6fc-45a6-8bbc-21eb39c98e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:26.695460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e19ce5-b6fc-45a6-8bbc-21eb39c98e9a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:26.695742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e19ce5-b6fc-45a6-8bbc-21eb39c98e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:26.696009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e19ce5-b6fc-45a6-8bbc-21eb39c98e9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:26.713442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e19ce5-b6fc-45a6-8bbc-21eb39c98e9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-06e19ce5-b6fc-45a6-8bbc-21eb39c98e9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:26.718930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:26.719857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:26.720700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:26.720700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:26.720700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:26.720700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:26.720700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:26.720700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:26.720700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:26.720700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:26.720700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:26.720700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:26.720700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:26.720700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:26.721594 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 312/615] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:26.728286 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e19ce5-b6fc-45a6-8bbc-21eb39c98e9a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:26.800149 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383a520-1a2f-4ec0-bab4-54ebb7a9afc1 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:26.865228 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86eb2c-ce68-43e4-84ab-6e34d8abc24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:26.865848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86eb2c-ce68-43e4-84ab-6e34d8abc24e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:26.866130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86eb2c-ce68-43e4-84ab-6e34d8abc24e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:26.866394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86eb2c-ce68-43e4-84ab-6e34d8abc24e None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:26.867102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86eb2c-ce68-43e4-84ab-6e34d8abc24e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:26.867395 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86eb2c-ce68-43e4-84ab-6e34d8abc24e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:26.868817 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86eb2c-ce68-43e4-84ab-6e34d8abc24e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:26.890389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 304/616] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:26.896067 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86eb2c-ce68-43e4-84ab-6e34d8abc24e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:26.920739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e19ce5-b6fc-45a6-8bbc-21eb39c98e9a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:27.002272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:27.010902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89446bfa-bb33-4e8c-9136-bc19535e0f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:27.011509 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89446bfa-bb33-4e8c-9136-bc19535e0f70 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:27.018606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89446bfa-bb33-4e8c-9136-bc19535e0f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:27.018606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89446bfa-bb33-4e8c-9136-bc19535e0f70 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:27.018606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89446bfa-bb33-4e8c-9136-bc19535e0f70 None admin] RBAC: Adding query filter params (name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:27.018606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89446bfa-bb33-4e8c-9136-bc19535e0f70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:27.018606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89446bfa-bb33-4e8c-9136-bc19535e0f70 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:27.033165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 313/617] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:27.050106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89446bfa-bb33-4e8c-9136-bc19535e0f70 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:27.053118 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86eb2c-ce68-43e4-84ab-6e34d8abc24e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:27.082187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:27.134330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522194d5-e2ba-427f-b919-0da07eb18a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:27.134875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522194d5-e2ba-427f-b919-0da07eb18a5c None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:27.135147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522194d5-e2ba-427f-b919-0da07eb18a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:27.135409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522194d5-e2ba-427f-b919-0da07eb18a5c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:27.152813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522194d5-e2ba-427f-b919-0da07eb18a5c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:27.154317 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522194d5-e2ba-427f-b919-0da07eb18a5c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-522194d5-e2ba-427f-b919-0da07eb18a5c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:27.168156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:27.169135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:27.170278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:27.170278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:27.170278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:27.170278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:27.170278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:27.170278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:27.170278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:27.170278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:27.170278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 05:24:27.170278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:27.174100 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 305/618] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:24:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:27.175579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522194d5-e2ba-427f-b919-0da07eb18a5c None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:27.241585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89446bfa-bb33-4e8c-9136-bc19535e0f70 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:27.289088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522194d5-e2ba-427f-b919-0da07eb18a5c None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:27.317261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:27.318910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d017e-168e-476a-843c-969d92e477e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:27.319550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d017e-168e-476a-843c-969d92e477e2 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:27.319827 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d017e-168e-476a-843c-969d92e477e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:27.320096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d017e-168e-476a-843c-969d92e477e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:27.326366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d5a21-d59b-490f-a6f7-51d670a5f366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:27.330018 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d5a21-d59b-490f-a6f7-51d670a5f366 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:27.330018 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d5a21-d59b-490f-a6f7-51d670a5f366 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:27.330018 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d5a21-d59b-490f-a6f7-51d670a5f366 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:27.330018 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d5a21-d59b-490f-a6f7-51d670a5f366 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:27.330018 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d5a21-d59b-490f-a6f7-51d670a5f366 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:27.331199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d5a21-d59b-490f-a6f7-51d670a5f366 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:27.346698 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d017e-168e-476a-843c-969d92e477e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:27.348141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d017e-168e-476a-843c-969d92e477e2 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:27.354088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 306/619] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Sep 16 05:24:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:27.355747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d5a21-d59b-490f-a6f7-51d670a5f366 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-275d017e-168e-476a-843c-969d92e477e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:27.378406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:27.379414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:27.380316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:27.380316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:27.380316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:27.386300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 314/620] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:24:27.395777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d017e-168e-476a-843c-969d92e477e2 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:27.468990 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d5a21-d59b-490f-a6f7-51d670a5f366 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:27.507811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9d377-c965-4b57-b891-103e79be3908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:27.508436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9d377-c965-4b57-b891-103e79be3908 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:27.508706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9d377-c965-4b57-b891-103e79be3908 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:27.508972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9d377-c965-4b57-b891-103e79be3908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:27.520286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9d377-c965-4b57-b891-103e79be3908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:27.521866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb9d377-c965-4b57-b891-103e79be3908 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-6cb9d377-c965-4b57-b891-103e79be3908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:27.537488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:27.538640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:27.539738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:27.539738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:27.539738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:27.544729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 307/621] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:24:27.545967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9d377-c965-4b57-b891-103e79be3908 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:27.598233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:27.605005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:27.676965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9d377-c965-4b57-b891-103e79be3908 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:27.688635 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d017e-168e-476a-843c-969d92e477e2 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:27.706188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64785d-dabf-42ca-a1f1-94cb785be6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:27.707266 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64785d-dabf-42ca-a1f1-94cb785be6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:27.707550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64785d-dabf-42ca-a1f1-94cb785be6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:27.708167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64785d-dabf-42ca-a1f1-94cb785be6e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:27.708840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64785d-dabf-42ca-a1f1-94cb785be6e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:27.709142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64785d-dabf-42ca-a1f1-94cb785be6e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:27.711235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64785d-dabf-42ca-a1f1-94cb785be6e7 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:27.730388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 308/622] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:27.731669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64785d-dabf-42ca-a1f1-94cb785be6e7 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:27.772129 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fbf55-85a6-4142-a4a3-4f302c7c9ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:27.772734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fbf55-85a6-4142-a4a3-4f302c7c9ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:27.773012 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fbf55-85a6-4142-a4a3-4f302c7c9ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:27.773281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fbf55-85a6-4142-a4a3-4f302c7c9ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:27.773957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fbf55-85a6-4142-a4a3-4f302c7c9ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:27.774256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fbf55-85a6-4142-a4a3-4f302c7c9ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:27.775682 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fbf55-85a6-4142-a4a3-4f302c7c9ec8 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:27.789122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 315/623] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:27.790444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1395c-ce27-40f8-901c-98b42cbf8c9c None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:27.790734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1395c-ce27-40f8-901c-98b42cbf8c9c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:27.791048 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1395c-ce27-40f8-901c-98b42cbf8c9c None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:27.797614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 316/624] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:27.798573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fcc46-a2e6-4848-87bd-120cdd0a8a5e None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:27.798910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fcc46-a2e6-4848-87bd-120cdd0a8a5e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:27.799202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fcc46-a2e6-4848-87bd-120cdd0a8a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:27.818264 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-597fcc46-a2e6-4848-87bd-120cdd0a8a5e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:27.819500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-597fcc46-a2e6-4848-87bd-120cdd0a8a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:27.850366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:27.858352 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64785d-dabf-42ca-a1f1-94cb785be6e7 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:27.894112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 317/625] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:27.895023 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fcc46-a2e6-4848-87bd-120cdd0a8a5e None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:27.899237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4ed7f-e649-4281-a1d0-6e5b7a4cc30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:27.899865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4ed7f-e649-4281-a1d0-6e5b7a4cc30f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:27.900137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4ed7f-e649-4281-a1d0-6e5b7a4cc30f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:27.900403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4ed7f-e649-4281-a1d0-6e5b7a4cc30f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:27.908664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4ed7f-e649-4281-a1d0-6e5b7a4cc30f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:27.910116 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4ed7f-e649-4281-a1d0-6e5b7a4cc30f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-83b4ed7f-e649-4281-a1d0-6e5b7a4cc30f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:27.922457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:27.923543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:27.924486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:27.924486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:27.924486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:27.924486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:27.924486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:27.924486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:27.924486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:27.924486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:27.934089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 309/626] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:27.934089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4ed7f-e649-4281-a1d0-6e5b7a4cc30f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:28.044397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4ed7f-e649-4281-a1d0-6e5b7a4cc30f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:28.082215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:28.086434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a9a39-274e-49eb-a902-109bc5131be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:28.087038 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a9a39-274e-49eb-a902-109bc5131be6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:28.087477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a9a39-274e-49eb-a902-109bc5131be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:28.087746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a9a39-274e-49eb-a902-109bc5131be6 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:28.088460 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a9a39-274e-49eb-a902-109bc5131be6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:28.088755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a9a39-274e-49eb-a902-109bc5131be6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:28.090308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a9a39-274e-49eb-a902-109bc5131be6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:28.109670 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 310/627] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:28.111056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a9a39-274e-49eb-a902-109bc5131be6 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:28.132487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:28.155988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fcc46-a2e6-4848-87bd-120cdd0a8a5e None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:28.195939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bf35c-ec53-4a59-aff8-e2e4fadf1c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:28.196485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bf35c-ec53-4a59-aff8-e2e4fadf1c49 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:28.196760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bf35c-ec53-4a59-aff8-e2e4fadf1c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:28.197028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bf35c-ec53-4a59-aff8-e2e4fadf1c49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:28.204460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bf35c-ec53-4a59-aff8-e2e4fadf1c49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:28.205930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bf35c-ec53-4a59-aff8-e2e4fadf1c49 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-326bf35c-ec53-4a59-aff8-e2e4fadf1c49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:28.218284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:28.219254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:28.220298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:28.220298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:28.220298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:28.220298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:28.220298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:28.220298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:28.220298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:28.220298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:28.220298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:24:28.220298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:28.220298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 318/628] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 16 05:24:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:28.221268 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e343654-6230-4c03-985d-fe4acd135958 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:28.221268 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e343654-6230-4c03-985d-fe4acd135958 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:28.221268 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e343654-6230-4c03-985d-fe4acd135958 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:28.225876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 319/629] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:28.225876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397c548-1e06-46d1-ada8-f67ff730ceae None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:28.225876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397c548-1e06-46d1-ada8-f67ff730ceae None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:28.225876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397c548-1e06-46d1-ada8-f67ff730ceae None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:28.225876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 320/630] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:28.230189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397c548-1e06-46d1-ada8-f67ff730ceae None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:28.235602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a9a39-274e-49eb-a902-109bc5131be6 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:28.273337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50763e-a441-4028-ab38-43a38d6ea6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:28.274024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50763e-a441-4028-ab38-43a38d6ea6b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:28.274293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50763e-a441-4028-ab38-43a38d6ea6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:28.274555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50763e-a441-4028-ab38-43a38d6ea6b4 None admin] PATH_INFO: `/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/b260fed0d99a4d3fa8f9c2ad79223219/roles/ee551384189f433bba7e06f47a794cf0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:28.289174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50763e-a441-4028-ab38-43a38d6ea6b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=29b2e27932984de79ee8dfbe95312ac2, user_id=b260fed0d99a4d3fa8f9c2ad79223219, role_id=ee551384189f433bba7e06f47a794cf0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:28.290633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50763e-a441-4028-ab38-43a38d6ea6b4 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:28.334378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397c548-1e06-46d1-ada8-f67ff730ceae None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:28.348689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:28.368529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a4f0c-bec5-417a-91e5-ea981de2c580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:28.369121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a4f0c-bec5-417a-91e5-ea981de2c580 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:28.369395 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a4f0c-bec5-417a-91e5-ea981de2c580 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:28.369660 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a4f0c-bec5-417a-91e5-ea981de2c580 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:28.370395 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a4f0c-bec5-417a-91e5-ea981de2c580 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:28.370693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a4f0c-bec5-417a-91e5-ea981de2c580 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:28.372087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a4f0c-bec5-417a-91e5-ea981de2c580 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:28.388413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 321/631] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 16 05:24:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:28.389527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8026976-31ab-4da6-8dc6-97cba00d2360 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:28.389806 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8026976-31ab-4da6-8dc6-97cba00d2360 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:28.390625 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8026976-31ab-4da6-8dc6-97cba00d2360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:28.392266 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 311/632] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:24:28 2022] PUT /identity/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/b260fed0d99a4d3fa8f9c2ad79223219/roles/ee551384189f433bba7e06f47a794cf0 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:24:28.406411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08abff-e327-4c0d-9ffd-ad7fd1d98bd5 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:28.406411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08abff-e327-4c0d-9ffd-ad7fd1d98bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:28.406411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce08abff-e327-4c0d-9ffd-ad7fd1d98bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:28.410678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-d8026976-31ab-4da6-8dc6-97cba00d2360 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:28.411911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d8026976-31ab-4da6-8dc6-97cba00d2360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:28.423150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-ce08abff-e327-4c0d-9ffd-ad7fd1d98bd5 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:28.424381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-ce08abff-e327-4c0d-9ffd-ad7fd1d98bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:28.489552 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 322/633] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:28.490809 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8026976-31ab-4da6-8dc6-97cba00d2360 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:28.502459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:28.513308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 312/634] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:28.514314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08abff-e327-4c0d-9ffd-ad7fd1d98bd5 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:28.657626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8026976-31ab-4da6-8dc6-97cba00d2360 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:28.659810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:28.667849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce08abff-e327-4c0d-9ffd-ad7fd1d98bd5 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:28.703585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841adff-2c7a-4ba3-a703-463d6253b78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:28.704218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841adff-2c7a-4ba3-a703-463d6253b78a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:28.704743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841adff-2c7a-4ba3-a703-463d6253b78a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:28.705016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841adff-2c7a-4ba3-a703-463d6253b78a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:28.711034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb23c0-764e-4130-811c-fc72d378005e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:28.711857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb23c0-764e-4130-811c-fc72d378005e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:28.712136 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb23c0-764e-4130-811c-fc72d378005e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:28.712484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb23c0-764e-4130-811c-fc72d378005e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:28.718170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c841adff-2c7a-4ba3-a703-463d6253b78a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-c841adff-2c7a-4ba3-a703-463d6253b78a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:28.724430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:28.725424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:28.726315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:28.726315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:28.726315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:28.726315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:28.726315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:28.726315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:28.726315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:28.726315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:28.726315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:28.726315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:28.726315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:28.726315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:28.728971 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 323/635] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:28.735153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841adff-2c7a-4ba3-a703-463d6253b78a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:28.837671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:28.845608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb23c0-764e-4130-811c-fc72d378005e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:28.847091 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb23c0-764e-4130-811c-fc72d378005e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:28.882005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841adff-2c7a-4ba3-a703-463d6253b78a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:28.916317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87071c5-9acb-469d-a5d3-43246cad01f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:28.916921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87071c5-9acb-469d-a5d3-43246cad01f0 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:28.917193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87071c5-9acb-469d-a5d3-43246cad01f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:28.917463 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87071c5-9acb-469d-a5d3-43246cad01f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:28.918162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87071c5-9acb-469d-a5d3-43246cad01f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:28.918462 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87071c5-9acb-469d-a5d3-43246cad01f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:28.919814 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87071c5-9acb-469d-a5d3-43246cad01f0 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:28.933194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 324/636] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:28.933194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87071c5-9acb-469d-a5d3-43246cad01f0 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:28.959775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:28.967075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 313/637] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:24:28.967075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb23c0-764e-4130-811c-fc72d378005e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:29.041790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87071c5-9acb-469d-a5d3-43246cad01f0 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:29.078262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:29.083168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9374e-f0bd-4789-87e9-047a5a74856c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:29.084201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9374e-f0bd-4789-87e9-047a5a74856c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:29.084631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9374e-f0bd-4789-87e9-047a5a74856c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:29.084903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da9374e-f0bd-4789-87e9-047a5a74856c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:29.086363 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb23c0-764e-4130-811c-fc72d378005e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:29.101222 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da9374e-f0bd-4789-87e9-047a5a74856c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-6da9374e-f0bd-4789-87e9-047a5a74856c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:29.107336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:29.108282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:29.109742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:29.109742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:29.109742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:29.109742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:29.109742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:29.109742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:29.109742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:29.109742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:29.109742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:29.109742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:29.109742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:29.109742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:29.109742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 325/638] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:29.111006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9374e-f0bd-4789-87e9-047a5a74856c None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:29.132193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaf9ac-45b2-41b7-bacf-1237d17aca86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:29.132821 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaf9ac-45b2-41b7-bacf-1237d17aca86 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:29.133328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaf9ac-45b2-41b7-bacf-1237d17aca86 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:29.133446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aaf9ac-45b2-41b7-bacf-1237d17aca86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:29.144085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aaf9ac-45b2-41b7-bacf-1237d17aca86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-c1aaf9ac-45b2-41b7-bacf-1237d17aca86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:29.149966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:29.150908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:29.151771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:29.151771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:29.151771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:29.151771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:29.151771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:29.151771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:29.151771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:29.151771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:29.151771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:29.151771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:29.151771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:29.151771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:29.151771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 314/639] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:29.153088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaf9ac-45b2-41b7-bacf-1237d17aca86 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:29.198566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:29.215322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da9374e-f0bd-4789-87e9-047a5a74856c None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:29.250891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca9cac-ba1d-4d95-bbb7-6b61fde1a0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:29.251821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca9cac-ba1d-4d95-bbb7-6b61fde1a0eb None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:29.252098 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca9cac-ba1d-4d95-bbb7-6b61fde1a0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:29.252507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfca9cac-ba1d-4d95-bbb7-6b61fde1a0eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:29.264707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca9cac-ba1d-4d95-bbb7-6b61fde1a0eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:29.266371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfca9cac-ba1d-4d95-bbb7-6b61fde1a0eb None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:29.269162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aaf9ac-45b2-41b7-bacf-1237d17aca86 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-dfca9cac-ba1d-4d95-bbb7-6b61fde1a0eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:29.281649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:29.282714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:29.283603 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:29.283603 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:29.283603 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:29.283603 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 326/640] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:29.284453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca9cac-ba1d-4d95-bbb7-6b61fde1a0eb None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:29.309813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd684a-b1c3-4bd4-8a89-a58c684d1e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:29.310762 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd684a-b1c3-4bd4-8a89-a58c684d1e05 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:29.311038 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd684a-b1c3-4bd4-8a89-a58c684d1e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:29.311304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd684a-b1c3-4bd4-8a89-a58c684d1e05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:29.322757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd684a-b1c3-4bd4-8a89-a58c684d1e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:29.324176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd684a-b1c3-4bd4-8a89-a58c684d1e05 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-37bd684a-b1c3-4bd4-8a89-a58c684d1e05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:29.338639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:29.339657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:29.340572 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:29.340572 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:29.340572 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:29.345020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 315/641] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:29.345797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd684a-b1c3-4bd4-8a89-a58c684d1e05 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:29.357331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:29.400908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfca9cac-ba1d-4d95-bbb7-6b61fde1a0eb None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:29.437412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc98dd6-39fb-4dd2-8a1b-9256368e39ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:29.438039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98dd6-39fb-4dd2-8a1b-9256368e39ff None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:29.438316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98dd6-39fb-4dd2-8a1b-9256368e39ff None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:29.438587 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc98dd6-39fb-4dd2-8a1b-9256368e39ff None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:29.439246 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc98dd6-39fb-4dd2-8a1b-9256368e39ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:29.439566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc98dd6-39fb-4dd2-8a1b-9256368e39ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:29.440919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc98dd6-39fb-4dd2-8a1b-9256368e39ff None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:29.443351 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:29.450695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 327/642] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:29.451311 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc98dd6-39fb-4dd2-8a1b-9256368e39ff None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:29.460953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd684a-b1c3-4bd4-8a89-a58c684d1e05 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:29.498765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0488f7-a1f6-4aa6-984b-48966c70b9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:29.499440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0488f7-a1f6-4aa6-984b-48966c70b9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:29.499975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0488f7-a1f6-4aa6-984b-48966c70b9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:29.499975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0488f7-a1f6-4aa6-984b-48966c70b9f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:29.500638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0488f7-a1f6-4aa6-984b-48966c70b9f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:29.500935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0488f7-a1f6-4aa6-984b-48966c70b9f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:29.502330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0488f7-a1f6-4aa6-984b-48966c70b9f7 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:29.510136 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 316/643] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:29.510918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0488f7-a1f6-4aa6-984b-48966c70b9f7 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:29.560187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc98dd6-39fb-4dd2-8a1b-9256368e39ff None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:29.590310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:29.596569 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c8043-3e95-42bf-b63a-c136a4532ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:29.597174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c8043-3e95-42bf-b63a-c136a4532ddc None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:29.597445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c8043-3e95-42bf-b63a-c136a4532ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:29.597712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c8043-3e95-42bf-b63a-c136a4532ddc None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:29.598371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c8043-3e95-42bf-b63a-c136a4532ddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:29.598662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c8043-3e95-42bf-b63a-c136a4532ddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:29.600010 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c8043-3e95-42bf-b63a-c136a4532ddc None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:29.616911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 328/644] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:29.616911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c8043-3e95-42bf-b63a-c136a4532ddc None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:29.623634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0488f7-a1f6-4aa6-984b-48966c70b9f7 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:29.662311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7fd31-8e84-4cc9-8950-66e5085f8e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:29.662959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7fd31-8e84-4cc9-8950-66e5085f8e2e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:29.663240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7fd31-8e84-4cc9-8950-66e5085f8e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:29.663495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7fd31-8e84-4cc9-8950-66e5085f8e2e None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:29.664131 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7fd31-8e84-4cc9-8950-66e5085f8e2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:29.664432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7fd31-8e84-4cc9-8950-66e5085f8e2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:29.665768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7fd31-8e84-4cc9-8950-66e5085f8e2e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:29.677136 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:29.682839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 317/645] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:29.682839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7fd31-8e84-4cc9-8950-66e5085f8e2e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:29.725777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c8043-3e95-42bf-b63a-c136a4532ddc None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:29.762325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b65103-3482-4e6f-9b52-abff2928685a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:29.762958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b65103-3482-4e6f-9b52-abff2928685a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:29.763221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b65103-3482-4e6f-9b52-abff2928685a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:29.763481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b65103-3482-4e6f-9b52-abff2928685a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:29.776191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b65103-3482-4e6f-9b52-abff2928685a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:29.777643 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b65103-3482-4e6f-9b52-abff2928685a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-f6b65103-3482-4e6f-9b52-abff2928685a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:29.792933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:29.793953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:29.795029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:29.795029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:29.795029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:29.795216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7fd31-8e84-4cc9-8950-66e5085f8e2e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:29.799288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 329/646] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:29.799936 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b65103-3482-4e6f-9b52-abff2928685a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:29.824683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:29.831944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413e9bd-7a4c-4d88-b997-f256d07adeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:29.836063 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413e9bd-7a4c-4d88-b997-f256d07adeb9 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:29.836063 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413e9bd-7a4c-4d88-b997-f256d07adeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:29.836063 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413e9bd-7a4c-4d88-b997-f256d07adeb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:29.843428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413e9bd-7a4c-4d88-b997-f256d07adeb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:29.844867 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413e9bd-7a4c-4d88-b997-f256d07adeb9 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-3413e9bd-7a4c-4d88-b997-f256d07adeb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:29.859902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:29.860853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:29.861750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:29.861750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:29.861750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:29.861750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 318/647] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:29.861750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413e9bd-7a4c-4d88-b997-f256d07adeb9 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:29.915594 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b65103-3482-4e6f-9b52-abff2928685a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:29.926631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:29.949567 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c27263-5cf1-433e-9ded-f00363f37e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:29.950208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c27263-5cf1-433e-9ded-f00363f37e02 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:29.950479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c27263-5cf1-433e-9ded-f00363f37e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:29.950902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c27263-5cf1-433e-9ded-f00363f37e02 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:29.958183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c27263-5cf1-433e-9ded-f00363f37e02 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:29.960159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c27263-5cf1-433e-9ded-f00363f37e02 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-99c27263-5cf1-433e-9ded-f00363f37e02 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:29.972921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:29.973904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:29.974884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:29.974884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:29.974884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:29.974884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:29.974884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:29.974884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:29.974884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:29.974884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:29.974884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 16 05:24:29.974884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:29.978673 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 330/648] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:29.979365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c27263-5cf1-433e-9ded-f00363f37e02 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:29.980963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413e9bd-7a4c-4d88-b997-f256d07adeb9 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:30.016436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d9b1b-814f-47f0-9fa0-0290e94e55e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:30.017325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d9b1b-814f-47f0-9fa0-0290e94e55e7 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:30.017599 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d9b1b-814f-47f0-9fa0-0290e94e55e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:30.017889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d9b1b-814f-47f0-9fa0-0290e94e55e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:30.025709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d9b1b-814f-47f0-9fa0-0290e94e55e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:30.027180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d9b1b-814f-47f0-9fa0-0290e94e55e7 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-138d9b1b-814f-47f0-9fa0-0290e94e55e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:30.041167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:30.042093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:30.042970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:30.042970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:30.042970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:30.042970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:30.042970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:30.042970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:30.042970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:30.042970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:30.042970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 319/649] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:30.043447 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d9b1b-814f-47f0-9fa0-0290e94e55e7 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:30.088269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:30.093930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c27263-5cf1-433e-9ded-f00363f37e02 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:30.135749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c87135-3262-40a7-84c9-d350f790e15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:30.136643 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c87135-3262-40a7-84c9-d350f790e15e None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:30.136933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c87135-3262-40a7-84c9-d350f790e15e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:30.137335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c87135-3262-40a7-84c9-d350f790e15e None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:30.138114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c87135-3262-40a7-84c9-d350f790e15e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:30.138414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c87135-3262-40a7-84c9-d350f790e15e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:30.141291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c87135-3262-40a7-84c9-d350f790e15e None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:30.146398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d9b1b-814f-47f0-9fa0-0290e94e55e7 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:30.158702 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 331/650] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:30.158702 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c87135-3262-40a7-84c9-d350f790e15e None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:30.182906 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0e7b1-bf43-42cb-a05a-d665567c588b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:30.183435 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0e7b1-bf43-42cb-a05a-d665567c588b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:30.183807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0e7b1-bf43-42cb-a05a-d665567c588b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:30.184080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea0e7b1-bf43-42cb-a05a-d665567c588b None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:30.184712 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0e7b1-bf43-42cb-a05a-d665567c588b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:30.185012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0e7b1-bf43-42cb-a05a-d665567c588b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:30.186410 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea0e7b1-bf43-42cb-a05a-d665567c588b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:30.190791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:30.204323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 320/651] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:30.204323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0e7b1-bf43-42cb-a05a-d665567c588b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:30.272610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c87135-3262-40a7-84c9-d350f790e15e None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:30.307871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08688e77-038f-4b2c-a45a-2072fa662f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:30.308600 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08688e77-038f-4b2c-a45a-2072fa662f36 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:30.308944 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08688e77-038f-4b2c-a45a-2072fa662f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:30.309284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08688e77-038f-4b2c-a45a-2072fa662f36 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:30.309375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea0e7b1-bf43-42cb-a05a-d665567c588b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:30.310122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08688e77-038f-4b2c-a45a-2072fa662f36 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:30.310490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08688e77-038f-4b2c-a45a-2072fa662f36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:30.311010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:30.312628 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08688e77-038f-4b2c-a45a-2072fa662f36 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:30.331196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 332/652] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Sep 16 05:24:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:30.332142 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08688e77-038f-4b2c-a45a-2072fa662f36 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:30.359713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fff1de-6fc6-4723-8bad-39160fbf4dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:30.360703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fff1de-6fc6-4723-8bad-39160fbf4dec None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:30.361023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fff1de-6fc6-4723-8bad-39160fbf4dec None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:30.361345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fff1de-6fc6-4723-8bad-39160fbf4dec None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:30.362076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fff1de-6fc6-4723-8bad-39160fbf4dec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:30.362379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fff1de-6fc6-4723-8bad-39160fbf4dec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:30.363790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fff1de-6fc6-4723-8bad-39160fbf4dec None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:30.380747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 321/653] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:30.380747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fff1de-6fc6-4723-8bad-39160fbf4dec None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:30.437526 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:30.449053 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08688e77-038f-4b2c-a45a-2072fa662f36 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:30.483382 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4366e0-0348-440a-b710-74b1dd2acce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:30.484544 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4366e0-0348-440a-b710-74b1dd2acce5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:30.484820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4366e0-0348-440a-b710-74b1dd2acce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:30.485089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4366e0-0348-440a-b710-74b1dd2acce5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:30.492118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4366e0-0348-440a-b710-74b1dd2acce5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:30.493569 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4366e0-0348-440a-b710-74b1dd2acce5 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:30.501260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fff1de-6fc6-4723-8bad-39160fbf4dec None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-ce4366e0-0348-440a-b710-74b1dd2acce5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:30.505811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:30.506961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:30.508130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:30.508130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:30.508130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:30.508130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:30.508130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:30.508130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:30.508130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:30.508130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:30.508130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 05:24:30.508130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:30.508130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 333/654] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:30.508130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4366e0-0348-440a-b710-74b1dd2acce5 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:30.542341 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70effc78-813f-4f7b-8bf6-224f13980d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:30.543062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70effc78-813f-4f7b-8bf6-224f13980d7d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:30.543346 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70effc78-813f-4f7b-8bf6-224f13980d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:30.543609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70effc78-813f-4f7b-8bf6-224f13980d7d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:30.551770 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70effc78-813f-4f7b-8bf6-224f13980d7d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:30.553181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70effc78-813f-4f7b-8bf6-224f13980d7d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-70effc78-813f-4f7b-8bf6-224f13980d7d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:30.566517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:30.567560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:30.568525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:30.568525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:30.568525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:30.568525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:30.568525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:30.568525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:30.568525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:30.568525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:30.568525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 16 05:24:30.568525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:30.568525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 322/655] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:30.569211 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70effc78-813f-4f7b-8bf6-224f13980d7d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:30.599557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:30.625249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4366e0-0348-440a-b710-74b1dd2acce5 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:30.660863 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f3812-faf2-41e6-80d4-fe3e8f4f447d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:30.661725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f3812-faf2-41e6-80d4-fe3e8f4f447d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:30.662052 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f3812-faf2-41e6-80d4-fe3e8f4f447d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:30.662319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f3812-faf2-41e6-80d4-fe3e8f4f447d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:30.679235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70effc78-813f-4f7b-8bf6-224f13980d7d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:30.681228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:30.721683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2a4d5-ec68-4b8d-b064-993ec0f44902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:30.722373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2a4d5-ec68-4b8d-b064-993ec0f44902 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:30.722649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2a4d5-ec68-4b8d-b064-993ec0f44902 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:30.722918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2a4d5-ec68-4b8d-b064-993ec0f44902 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:30.723628 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2a4d5-ec68-4b8d-b064-993ec0f44902 None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=7db75c20e9744feb8b5dc78c554f4977) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:30.723928 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2a4d5-ec68-4b8d-b064-993ec0f44902 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:30.725344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2a4d5-ec68-4b8d-b064-993ec0f44902 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:30.731273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 323/656] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:30 2022] GET /identity/v3/role_assignments?user.id=7db75c20e9744feb8b5dc78c554f4977&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:30.732040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2a4d5-ec68-4b8d-b064-993ec0f44902 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:30.735163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f3812-faf2-41e6-80d4-fe3e8f4f447d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:30.736689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f3812-faf2-41e6-80d4-fe3e8f4f447d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:30.829563 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:30.846553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2a4d5-ec68-4b8d-b064-993ec0f44902 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:30.847213 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 334/657] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:24:30.848044 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f3812-faf2-41e6-80d4-fe3e8f4f447d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:30.891829 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ea2c7-46bf-434b-9359-8c1a2af6d206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:30.892451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ea2c7-46bf-434b-9359-8c1a2af6d206 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:30.892724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ea2c7-46bf-434b-9359-8c1a2af6d206 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:30.892994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ea2c7-46bf-434b-9359-8c1a2af6d206 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:30.893725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ea2c7-46bf-434b-9359-8c1a2af6d206 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:30.894047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ea2c7-46bf-434b-9359-8c1a2af6d206 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:30.895482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ea2c7-46bf-434b-9359-8c1a2af6d206 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:30.903835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:30.916626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 324/658] 10.222.0.49 () {56 vars in 1137 bytes} [Fri Sep 16 05:24:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:30.918439 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cece77d-062f-4c52-937c-08fb5fa844a7 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:30.918790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cece77d-062f-4c52-937c-08fb5fa844a7 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:30.919155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cece77d-062f-4c52-937c-08fb5fa844a7 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:30.920929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 325/659] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:30.922025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cece77d-062f-4c52-937c-08fb5fa844a7 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:30.967462 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f3812-faf2-41e6-80d4-fe3e8f4f447d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:31.009721 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66650f-1f02-497f-8bb1-1837948f58ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:31.010276 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66650f-1f02-497f-8bb1-1837948f58ef None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:31.010547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66650f-1f02-497f-8bb1-1837948f58ef None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:31.010808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc66650f-1f02-497f-8bb1-1837948f58ef None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:31.011527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66650f-1f02-497f-8bb1-1837948f58ef None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:31.011826 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66650f-1f02-497f-8bb1-1837948f58ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:31.013220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc66650f-1f02-497f-8bb1-1837948f58ef None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:31.032074 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 335/660] 10.222.0.49 () {56 vars in 1139 bytes} [Fri Sep 16 05:24:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:31.033033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66650f-1f02-497f-8bb1-1837948f58ef None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:31.040683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cece77d-062f-4c52-937c-08fb5fa844a7 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:31.079381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b8971-bcd9-4af9-9bcf-f5f5a10eb018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:31.080013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b8971-bcd9-4af9-9bcf-f5f5a10eb018 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:31.080293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b8971-bcd9-4af9-9bcf-f5f5a10eb018 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:31.080564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b8971-bcd9-4af9-9bcf-f5f5a10eb018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:31.090982 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b8971-bcd9-4af9-9bcf-f5f5a10eb018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:31.092409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b8971-bcd9-4af9-9bcf-f5f5a10eb018 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:31.095062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-f30b8971-bcd9-4af9-9bcf-f5f5a10eb018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:31.107909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:31.109567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:31.110502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:31.110502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:31.110502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:31.110502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 326/661] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:31.111405 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af37e8-e9b7-4873-959d-5f2d55d51102 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:31.111916 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af37e8-e9b7-4873-959d-5f2d55d51102 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:31.112248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af37e8-e9b7-4873-959d-5f2d55d51102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:31.130145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-32af37e8-e9b7-4873-959d-5f2d55d51102 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:31.131319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-32af37e8-e9b7-4873-959d-5f2d55d51102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:31.149235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc66650f-1f02-497f-8bb1-1837948f58ef None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:31.157103 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:31.188282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad917b9f-c881-4ced-8317-0486c9845ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:31.189143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad917b9f-c881-4ced-8317-0486c9845ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:31.189278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad917b9f-c881-4ced-8317-0486c9845ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:31.189494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad917b9f-c881-4ced-8317-0486c9845ce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:31.209992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 327/662] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:31.209992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af37e8-e9b7-4873-959d-5f2d55d51102 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:31.281570 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad917b9f-c881-4ced-8317-0486c9845ce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:31.283711 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad917b9f-c881-4ced-8317-0486c9845ce4 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:31.327493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af37e8-e9b7-4873-959d-5f2d55d51102 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:31.359431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:31.368840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d9497-ace8-4fd9-8ac5-2a1b73f4a076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:31.369448 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d9497-ace8-4fd9-8ac5-2a1b73f4a076 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:31.369716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d9497-ace8-4fd9-8ac5-2a1b73f4a076 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:31.381110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d9497-ace8-4fd9-8ac5-2a1b73f4a076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:31.418011 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 336/663] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:24:31.419043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad917b9f-c881-4ced-8317-0486c9845ce4 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:31.428732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:31.454488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d9497-ace8-4fd9-8ac5-2a1b73f4a076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:31.455978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d9497-ace8-4fd9-8ac5-2a1b73f4a076 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:31.536634 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad917b9f-c881-4ced-8317-0486c9845ce4 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:31.571138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 328/664] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:24:31.571920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb869b7-7afb-43c3-8ba6-278dd2b20b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:31.572525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb869b7-7afb-43c3-8ba6-278dd2b20b29 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:31.572802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb869b7-7afb-43c3-8ba6-278dd2b20b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:31.573068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb869b7-7afb-43c3-8ba6-278dd2b20b29 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:31.573570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d9497-ace8-4fd9-8ac5-2a1b73f4a076 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:31.573687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb869b7-7afb-43c3-8ba6-278dd2b20b29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:31.574024 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb869b7-7afb-43c3-8ba6-278dd2b20b29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:31.575581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb869b7-7afb-43c3-8ba6-278dd2b20b29 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:31.586959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 337/665] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:31.591925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb869b7-7afb-43c3-8ba6-278dd2b20b29 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:31.604124 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:31.656934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:31.681658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d9497-ace8-4fd9-8ac5-2a1b73f4a076 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:31.690971 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb869b7-7afb-43c3-8ba6-278dd2b20b29 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:31.720717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7ab35-b505-420d-8c57-1c581f916906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:31.721324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7ab35-b505-420d-8c57-1c581f916906 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:31.721591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7ab35-b505-420d-8c57-1c581f916906 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:31.721875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7ab35-b505-420d-8c57-1c581f916906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:31.730910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211807ad-a286-4201-8d88-a0b1028e4983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:31.731628 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211807ad-a286-4201-8d88-a0b1028e4983 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:31.731998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211807ad-a286-4201-8d88-a0b1028e4983 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:31.732373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211807ad-a286-4201-8d88-a0b1028e4983 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:31.733210 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7ab35-b505-420d-8c57-1c581f916906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:31.735935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7ab35-b505-420d-8c57-1c581f916906 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:31.743380 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211807ad-a286-4201-8d88-a0b1028e4983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-211807ad-a286-4201-8d88-a0b1028e4983 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:31.748713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:31.750429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:31.751467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:31.751467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:31.751467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:31.751467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:31.751467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:31.751467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:31.751467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:31.751467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:31.751467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:31.751467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:31.751467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:31.751467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-d8e7ab35-b505-420d-8c57-1c581f916906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:31.752253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:31.753443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:31.754545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:31.754545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:31.754545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:31.754545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 329/666] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:31.754545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7ab35-b505-420d-8c57-1c581f916906 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:31.761876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 338/667] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:31.761876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211807ad-a286-4201-8d88-a0b1028e4983 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:31.838443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:31.860017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211807ad-a286-4201-8d88-a0b1028e4983 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:31.861786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7ab35-b505-420d-8c57-1c581f916906 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:31.885289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:31.894865 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f84e3-6d8a-457d-b7a0-ce5f45625dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:31.896002 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f84e3-6d8a-457d-b7a0-ce5f45625dec None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:31.896002 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f84e3-6d8a-457d-b7a0-ce5f45625dec None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:31.896161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f84e3-6d8a-457d-b7a0-ce5f45625dec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:31.897804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34341afe-ea34-4bf8-9d0d-3512cedb9d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:31.898406 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34341afe-ea34-4bf8-9d0d-3512cedb9d04 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:31.898861 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34341afe-ea34-4bf8-9d0d-3512cedb9d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:31.898966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34341afe-ea34-4bf8-9d0d-3512cedb9d04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:31.902743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f84e3-6d8a-457d-b7a0-ce5f45625dec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:31.904183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f84e3-6d8a-457d-b7a0-ce5f45625dec None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:31.909979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34341afe-ea34-4bf8-9d0d-3512cedb9d04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:31.913429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34341afe-ea34-4bf8-9d0d-3512cedb9d04 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-579f84e3-6d8a-457d-b7a0-ce5f45625dec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:31.916460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:31.917357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:31.918292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:31.918292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:31.918292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:31.918292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:31.918292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:31.918292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:31.918292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:31.918292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:31.918292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 339/668] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:31.918787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f84e3-6d8a-457d-b7a0-ce5f45625dec None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-34341afe-ea34-4bf8-9d0d-3512cedb9d04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:31.937854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:31.938840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:31.944499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:31.944499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:31.944499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:31.944499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 330/669] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:31.944499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34341afe-ea34-4bf8-9d0d-3512cedb9d04 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:32.043595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f84e3-6d8a-457d-b7a0-ce5f45625dec None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:32.048439 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34341afe-ea34-4bf8-9d0d-3512cedb9d04 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:32.078800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:32.083892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365f66a-1417-496e-b91f-c437713215a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:32.084496 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365f66a-1417-496e-b91f-c437713215a8 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:32.084727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec0016-7db6-41a2-8e88-b69e5d7f05da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:32.084866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365f66a-1417-496e-b91f-c437713215a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:32.085027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365f66a-1417-496e-b91f-c437713215a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:32.085323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec0016-7db6-41a2-8e88-b69e5d7f05da None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:32.085600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec0016-7db6-41a2-8e88-b69e5d7f05da None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:32.085692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365f66a-1417-496e-b91f-c437713215a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:32.085893 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec0016-7db6-41a2-8e88-b69e5d7f05da None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:32.085989 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365f66a-1417-496e-b91f-c437713215a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:32.086537 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec0016-7db6-41a2-8e88-b69e5d7f05da None admin] RBAC: Adding query filter params (name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:32.086831 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec0016-7db6-41a2-8e88-b69e5d7f05da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:32.087335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365f66a-1417-496e-b91f-c437713215a8 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:32.088262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec0016-7db6-41a2-8e88-b69e5d7f05da None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:32.103679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 331/670] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:32.103679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec0016-7db6-41a2-8e88-b69e5d7f05da None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:32.103918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 340/671] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:32.103918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365f66a-1417-496e-b91f-c437713215a8 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:32.156850 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:32.210530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec0016-7db6-41a2-8e88-b69e5d7f05da None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:32.211429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365f66a-1417-496e-b91f-c437713215a8 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:32.247640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2be2e-7377-4521-afd1-5434c638ad38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:32.247896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9a7f0-3f06-423a-9e36-e88e8f045817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:32.248413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2be2e-7377-4521-afd1-5434c638ad38 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:32.248618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9a7f0-3f06-423a-9e36-e88e8f045817 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:32.248699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2be2e-7377-4521-afd1-5434c638ad38 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:32.248780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9a7f0-3f06-423a-9e36-e88e8f045817 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:32.248965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9a7f0-3f06-423a-9e36-e88e8f045817 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:32.249058 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2be2e-7377-4521-afd1-5434c638ad38 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:32.250192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9a7f0-3f06-423a-9e36-e88e8f045817 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:32.250487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9a7f0-3f06-423a-9e36-e88e8f045817 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:32.250584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2be2e-7377-4521-afd1-5434c638ad38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:32.250788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2be2e-7377-4521-afd1-5434c638ad38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:32.251889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9a7f0-3f06-423a-9e36-e88e8f045817 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:32.252130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2be2e-7377-4521-afd1-5434c638ad38 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:32.269078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 341/672] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:32.269078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2be2e-7377-4521-afd1-5434c638ad38 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:32.269296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 332/673] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:32.270561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9a7f0-3f06-423a-9e36-e88e8f045817 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:32.327885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:32.382296 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2be2e-7377-4521-afd1-5434c638ad38 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:32.389969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:32.406817 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9a7f0-3f06-423a-9e36-e88e8f045817 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:32.417430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47c1d-cb0b-4d3d-8960-fc515452fb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:32.418408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47c1d-cb0b-4d3d-8960-fc515452fb6a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:32.418900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47c1d-cb0b-4d3d-8960-fc515452fb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:32.419362 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a47c1d-cb0b-4d3d-8960-fc515452fb6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:32.432273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a47c1d-cb0b-4d3d-8960-fc515452fb6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:32.434228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a47c1d-cb0b-4d3d-8960-fc515452fb6a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:32.441327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cde92-e04a-4461-8aac-f87b3c2a0120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:32.441985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cde92-e04a-4461-8aac-f87b3c2a0120 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:32.442265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cde92-e04a-4461-8aac-f87b3c2a0120 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:32.442535 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cde92-e04a-4461-8aac-f87b3c2a0120 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:32.449088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cde92-e04a-4461-8aac-f87b3c2a0120 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-62a47c1d-cb0b-4d3d-8960-fc515452fb6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:32.450117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:32.451016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:32.451935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:32.451935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:32.451935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:32.452102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48cde92-e04a-4461-8aac-f87b3c2a0120 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:32.453043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 342/674] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:32.454207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47c1d-cb0b-4d3d-8960-fc515452fb6a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-c48cde92-e04a-4461-8aac-f87b3c2a0120 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:32.467957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:32.469353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:32.470684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:32.470684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:32.470684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:32.470684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:32.470684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:32.470684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:32.470684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:32.470684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:32.470684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 333/675] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:32.470684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cde92-e04a-4461-8aac-f87b3c2a0120 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:32.563290 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a47c1d-cb0b-4d3d-8960-fc515452fb6a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:32.582452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cde92-e04a-4461-8aac-f87b3c2a0120 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:32.597517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:32.598959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6172143-3af3-4737-b399-0308e591729e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:32.599915 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6172143-3af3-4737-b399-0308e591729e None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:32.600196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6172143-3af3-4737-b399-0308e591729e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:32.600471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6172143-3af3-4737-b399-0308e591729e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:32.610121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6172143-3af3-4737-b399-0308e591729e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:32.611568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6172143-3af3-4737-b399-0308e591729e None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:32.619843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286423f6-80e0-4503-aa50-57a584572fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:32.620684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286423f6-80e0-4503-aa50-57a584572fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:32.621152 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286423f6-80e0-4503-aa50-57a584572fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:32.621612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286423f6-80e0-4503-aa50-57a584572fb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:32.622569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286423f6-80e0-4503-aa50-57a584572fb3 None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=30d96ec6235843c5b67d87a3d0ff0f8f) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-e6172143-3af3-4737-b399-0308e591729e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:32.623014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:32.623878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:32.624834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:32.624834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:32.624834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:32.624834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:32.624834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:32.624834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:32.624834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:32.624834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:32.626086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286423f6-80e0-4503-aa50-57a584572fb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:32.628125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286423f6-80e0-4503-aa50-57a584572fb3 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:32.629167 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 343/676] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:32.630019 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6172143-3af3-4737-b399-0308e591729e None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:32.635980 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 334/677] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:32 2022] GET /identity/v3/role_assignments?user.id=30d96ec6235843c5b67d87a3d0ff0f8f&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:32.636957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286423f6-80e0-4503-aa50-57a584572fb3 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:32.652877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:32.745203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6172143-3af3-4737-b399-0308e591729e None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:32.788007 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286423f6-80e0-4503-aa50-57a584572fb3 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:32.792877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0152364-e6a2-41dd-9b11-729ffb0e4f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:32.793878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0152364-e6a2-41dd-9b11-729ffb0e4f41 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:32.794379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0152364-e6a2-41dd-9b11-729ffb0e4f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:32.794872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0152364-e6a2-41dd-9b11-729ffb0e4f41 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:32.795731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0152364-e6a2-41dd-9b11-729ffb0e4f41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:32.796457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0152364-e6a2-41dd-9b11-729ffb0e4f41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:32.798164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0152364-e6a2-41dd-9b11-729ffb0e4f41 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:32.812417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 344/678] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:32.813605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0152364-e6a2-41dd-9b11-729ffb0e4f41 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:32.829216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e4be1-4a4b-4cc3-932f-fdfd07126eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:32.829870 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e4be1-4a4b-4cc3-932f-fdfd07126eef None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:32.830152 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e4be1-4a4b-4cc3-932f-fdfd07126eef None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:32.830472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e4be1-4a4b-4cc3-932f-fdfd07126eef None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:32.831169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e4be1-4a4b-4cc3-932f-fdfd07126eef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:32.831469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e4be1-4a4b-4cc3-932f-fdfd07126eef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:32.832967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e4be1-4a4b-4cc3-932f-fdfd07126eef None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:32.847786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 335/679] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:32.848671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e4be1-4a4b-4cc3-932f-fdfd07126eef None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:32.889517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:32.901414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:32.932931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0152364-e6a2-41dd-9b11-729ffb0e4f41 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:32.982135 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb394409-58ca-422d-aa26-79ebb25c7e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:32.982699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb394409-58ca-422d-aa26-79ebb25c7e23 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:32.982994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb394409-58ca-422d-aa26-79ebb25c7e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:32.983250 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb394409-58ca-422d-aa26-79ebb25c7e23 None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:32.984053 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb394409-58ca-422d-aa26-79ebb25c7e23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:32.984359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb394409-58ca-422d-aa26-79ebb25c7e23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:32.985786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb394409-58ca-422d-aa26-79ebb25c7e23 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:32.986042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e4be1-4a4b-4cc3-932f-fdfd07126eef None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:32.999972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 345/680] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:33.001431 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb394409-58ca-422d-aa26-79ebb25c7e23 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:33.027999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789beae5-e75a-4618-9746-dc08c8d46ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:33.028675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789beae5-e75a-4618-9746-dc08c8d46ebc None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:33.028953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789beae5-e75a-4618-9746-dc08c8d46ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:33.029220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789beae5-e75a-4618-9746-dc08c8d46ebc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:33.039357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789beae5-e75a-4618-9746-dc08c8d46ebc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:33.040788 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789beae5-e75a-4618-9746-dc08c8d46ebc None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-789beae5-e75a-4618-9746-dc08c8d46ebc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:33.053111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:33.054205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:33.055143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:33.055143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:33.055143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:33.055143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:33.055143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:33.055143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:33.055143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:33.055143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:33.055143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 05:24:33.055143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:33.055143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 336/681] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:24:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:33.055758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789beae5-e75a-4618-9746-dc08c8d46ebc None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:33.110775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb394409-58ca-422d-aa26-79ebb25c7e23 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:33.144666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b8faae-66db-405c-a635-f4a428fe4d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:33.145192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8faae-66db-405c-a635-f4a428fe4d74 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:33.145469 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8faae-66db-405c-a635-f4a428fe4d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:33.145739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b8faae-66db-405c-a635-f4a428fe4d74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:33.146490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8faae-66db-405c-a635-f4a428fe4d74 None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=ff50e50587254bac912fb342a7e09891) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:33.146802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8faae-66db-405c-a635-f4a428fe4d74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:33.148222 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b8faae-66db-405c-a635-f4a428fe4d74 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:33.153959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 346/682] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:33 2022] GET /identity/v3/role_assignments?user.id=ff50e50587254bac912fb342a7e09891&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:33.155124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f922c5-7995-44a5-a5f6-1f79b278e9d4 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:33.155455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f922c5-7995-44a5-a5f6-1f79b278e9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:33.155734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f922c5-7995-44a5-a5f6-1f79b278e9d4 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:33.156950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 347/683] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:33.158672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f922c5-7995-44a5-a5f6-1f79b278e9d4 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:33.163605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:33.172357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:33.181146 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789beae5-e75a-4618-9746-dc08c8d46ebc None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:33.220936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00a6d1-aada-445a-a06d-6940a013a6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:33.221164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00a6d1-aada-445a-a06d-6940a013a6fd None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:33.221406 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00a6d1-aada-445a-a06d-6940a013a6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:33.221654 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00a6d1-aada-445a-a06d-6940a013a6fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:33.222458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00a6d1-aada-445a-a06d-6940a013a6fd None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=3e923be9aeed43728690d24d85dbe062) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:33.222762 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00a6d1-aada-445a-a06d-6940a013a6fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:33.224366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00a6d1-aada-445a-a06d-6940a013a6fd None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:33.231852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 337/684] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:33 2022] GET /identity/v3/role_assignments?user.id=3e923be9aeed43728690d24d85dbe062&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:33.232880 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ef29a-9c99-4d92-99b3-a44aeb2bf32e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:33.233163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ef29a-9c99-4d92-99b3-a44aeb2bf32e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:33.233434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ef29a-9c99-4d92-99b3-a44aeb2bf32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:33.255264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-430ef29a-9c99-4d92-99b3-a44aeb2bf32e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:33.256562 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-430ef29a-9c99-4d92-99b3-a44aeb2bf32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:33.303525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f922c5-7995-44a5-a5f6-1f79b278e9d4 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:33.341359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87503ff4-1ece-41d2-8ac6-bb4f1b815427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:33.342088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87503ff4-1ece-41d2-8ac6-bb4f1b815427 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:33.342369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87503ff4-1ece-41d2-8ac6-bb4f1b815427 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:33.342641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87503ff4-1ece-41d2-8ac6-bb4f1b815427 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:33.343573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87503ff4-1ece-41d2-8ac6-bb4f1b815427 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:33.343869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87503ff4-1ece-41d2-8ac6-bb4f1b815427 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:33.345407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87503ff4-1ece-41d2-8ac6-bb4f1b815427 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:33.353915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 338/685] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:33.393163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 348/686] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Sep 16 05:24:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:33.398756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ef29a-9c99-4d92-99b3-a44aeb2bf32e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:33.503011 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:33.513355 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ef29a-9c99-4d92-99b3-a44aeb2bf32e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:33.553994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd4f75-015f-48c2-bce7-912d0dce6d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:33.554969 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd4f75-015f-48c2-bce7-912d0dce6d95 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:33.555443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd4f75-015f-48c2-bce7-912d0dce6d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:33.555940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dd4f75-015f-48c2-bce7-912d0dce6d95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:33.631717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd4f75-015f-48c2-bce7-912d0dce6d95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:33.633945 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dd4f75-015f-48c2-bce7-912d0dce6d95 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:33.659809 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87503ff4-1ece-41d2-8ac6-bb4f1b815427 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:33.741401 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:33.752071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:33.766295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 339/687] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:24:33.769978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd4f75-015f-48c2-bce7-912d0dce6d95 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:33.824322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87503ff4-1ece-41d2-8ac6-bb4f1b815427 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:33.895293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b878d4c-e29e-441f-89a0-4abc77f92890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:33.896029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b878d4c-e29e-441f-89a0-4abc77f92890 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:33.896303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b878d4c-e29e-441f-89a0-4abc77f92890 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:33.896568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b878d4c-e29e-441f-89a0-4abc77f92890 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:33.927877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b878d4c-e29e-441f-89a0-4abc77f92890 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:33.930715 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dd4f75-015f-48c2-bce7-912d0dce6d95 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-1b878d4c-e29e-441f-89a0-4abc77f92890 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:33.933218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:33.934176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:33.935022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:33.935022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:33.935022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:33.935022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:33.935022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:33.935022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:33.935022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:33.935022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:33.935022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:33.935022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:33.935022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:33.935022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:33.942384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 349/688] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:24:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:33.948906 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b878d4c-e29e-441f-89a0-4abc77f92890 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:33.977090 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da703f0-7ac0-4339-91c8-67ea523d4a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:33.977725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da703f0-7ac0-4339-91c8-67ea523d4a12 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:33.978028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da703f0-7ac0-4339-91c8-67ea523d4a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:33.978360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da703f0-7ac0-4339-91c8-67ea523d4a12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:33.994352 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da703f0-7ac0-4339-91c8-67ea523d4a12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:33.995818 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da703f0-7ac0-4339-91c8-67ea523d4a12 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-6da703f0-7ac0-4339-91c8-67ea523d4a12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:34.014343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:34.016056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:34.017215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:34.017215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:34.017215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:34.019438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 340/689] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:24:34.024766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da703f0-7ac0-4339-91c8-67ea523d4a12 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:34.114200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:34.188204 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b878d4c-e29e-441f-89a0-4abc77f92890 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:34.232887 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da703f0-7ac0-4339-91c8-67ea523d4a12 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:34.235495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:34.264122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc3a95-9fc8-4ec9-93d2-36efcce676fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:34.264765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc3a95-9fc8-4ec9-93d2-36efcce676fe None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:34.265014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc3a95-9fc8-4ec9-93d2-36efcce676fe None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:34.265393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc3a95-9fc8-4ec9-93d2-36efcce676fe None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:34.266272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc3a95-9fc8-4ec9-93d2-36efcce676fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:34.266824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc3a95-9fc8-4ec9-93d2-36efcce676fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:34.268199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc3a95-9fc8-4ec9-93d2-36efcce676fe None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:34.293754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 350/690] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:34.297387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc3a95-9fc8-4ec9-93d2-36efcce676fe None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:34.308338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e51b96-8e79-47c6-9d1f-1730b1ffdd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:34.308953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e51b96-8e79-47c6-9d1f-1730b1ffdd5d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:34.309221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e51b96-8e79-47c6-9d1f-1730b1ffdd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:34.309481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e51b96-8e79-47c6-9d1f-1730b1ffdd5d None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:34.310154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e51b96-8e79-47c6-9d1f-1730b1ffdd5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:34.310445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e51b96-8e79-47c6-9d1f-1730b1ffdd5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:34.311884 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e51b96-8e79-47c6-9d1f-1730b1ffdd5d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:34.339636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 341/691] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:34.346111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e51b96-8e79-47c6-9d1f-1730b1ffdd5d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:34.455837 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc3a95-9fc8-4ec9-93d2-36efcce676fe None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:34.496381 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb2a35-cad6-41b3-b2e3-6b8a11b41417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:34.497003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb2a35-cad6-41b3-b2e3-6b8a11b41417 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:34.497274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb2a35-cad6-41b3-b2e3-6b8a11b41417 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:34.497538 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb2a35-cad6-41b3-b2e3-6b8a11b41417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:34.526372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb2a35-cad6-41b3-b2e3-6b8a11b41417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:34.527913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb2a35-cad6-41b3-b2e3-6b8a11b41417 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-e7bb2a35-cad6-41b3-b2e3-6b8a11b41417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:34.559127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:34.560249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:34.561113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:34.561113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:34.561113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:34.568093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e51b96-8e79-47c6-9d1f-1730b1ffdd5d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:34.569991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 351/692] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:34.570506 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:34.570875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb2a35-cad6-41b3-b2e3-6b8a11b41417 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:34.639937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ff8d1-673a-4572-8cf9-82863e62ed25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:34.642073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:34.642220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ff8d1-673a-4572-8cf9-82863e62ed25 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:34.642403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ff8d1-673a-4572-8cf9-82863e62ed25 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:34.642672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ff8d1-673a-4572-8cf9-82863e62ed25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:34.657339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ff8d1-673a-4572-8cf9-82863e62ed25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:34.660051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ff8d1-673a-4572-8cf9-82863e62ed25 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-793ff8d1-673a-4572-8cf9-82863e62ed25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:34.681528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:34.682472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:34.683621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:34.683621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:34.683621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:34.683621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:34.683621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:34.683621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:34.683621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:34.683621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:34.683621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 342/693] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:34.691074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ff8d1-673a-4572-8cf9-82863e62ed25 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:34.779327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb2a35-cad6-41b3-b2e3-6b8a11b41417 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:34.804251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ff8d1-673a-4572-8cf9-82863e62ed25 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:34.840280 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbb6cf-4938-494f-8499-70c7e86e8d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:34.840935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbb6cf-4938-494f-8499-70c7e86e8d78 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:34.841293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbb6cf-4938-494f-8499-70c7e86e8d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:34.841562 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dbb6cf-4938-494f-8499-70c7e86e8d78 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:34.842278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbb6cf-4938-494f-8499-70c7e86e8d78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:34.842581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbb6cf-4938-494f-8499-70c7e86e8d78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:34.844051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dbb6cf-4938-494f-8499-70c7e86e8d78 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:34.856343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 343/694] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:34.860073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3211540-a070-4f2c-949c-a24229749a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:34.860811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3211540-a070-4f2c-949c-a24229749a60 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:34.861190 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3211540-a070-4f2c-949c-a24229749a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:34.861561 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3211540-a070-4f2c-949c-a24229749a60 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:34.862344 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3211540-a070-4f2c-949c-a24229749a60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:34.863617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbb6cf-4938-494f-8499-70c7e86e8d78 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:34.864044 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3211540-a070-4f2c-949c-a24229749a60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:34.865412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3211540-a070-4f2c-949c-a24229749a60 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:34.897724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 352/695] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:34.906097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3211540-a070-4f2c-949c-a24229749a60 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:34.914165 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:35.061072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dbb6cf-4938-494f-8499-70c7e86e8d78 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:35.080641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:35.090373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3211540-a070-4f2c-949c-a24229749a60 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:35.114347 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277d533-f489-4d39-b30b-66b1bca0dd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:35.115656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277d533-f489-4d39-b30b-66b1bca0dd1f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:35.115767 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277d533-f489-4d39-b30b-66b1bca0dd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:35.116015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277d533-f489-4d39-b30b-66b1bca0dd1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:35.117014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277d533-f489-4d39-b30b-66b1bca0dd1f None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=b260fed0d99a4d3fa8f9c2ad79223219) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:35.117319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277d533-f489-4d39-b30b-66b1bca0dd1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:35.118777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277d533-f489-4d39-b30b-66b1bca0dd1f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:35.126265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 344/696] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:34 2022] GET /identity/v3/role_assignments?user.id=b260fed0d99a4d3fa8f9c2ad79223219&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:35.127378 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4caf826-df36-460e-b3c0-48d8f4dfe62f None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:35.127656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4caf826-df36-460e-b3c0-48d8f4dfe62f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:35.127929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4caf826-df36-460e-b3c0-48d8f4dfe62f None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:35.130122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847ac44-284d-4b26-b1c6-3d1c3b3c18f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:35.130772 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847ac44-284d-4b26-b1c6-3d1c3b3c18f9 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:35.131112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847ac44-284d-4b26-b1c6-3d1c3b3c18f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:35.131386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847ac44-284d-4b26-b1c6-3d1c3b3c18f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:35.133786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 345/697] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:35.135837 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d6c57-c914-4c07-bcb6-ca8959b2011d None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:35.136115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d6c57-c914-4c07-bcb6-ca8959b2011d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:35.136382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d6c57-c914-4c07-bcb6-ca8959b2011d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:35.140071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847ac44-284d-4b26-b1c6-3d1c3b3c18f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:35.141972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847ac44-284d-4b26-b1c6-3d1c3b3c18f9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-c847ac44-284d-4b26-b1c6-3d1c3b3c18f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:35.156704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:35.160169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:35.161100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:35.161100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:35.161100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:35.161100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:35.161100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:35.161100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:35.161100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:35.161100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:35.161100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:24:35.161100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:35.161642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-5f3d6c57-c914-4c07-bcb6-ca8959b2011d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:35.161737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 353/698] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 16 05:24:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:35.161809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5f3d6c57-c914-4c07-bcb6-ca8959b2011d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:35.163731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847ac44-284d-4b26-b1c6-3d1c3b3c18f9 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:35.245551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 346/699] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:35.294943 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847ac44-284d-4b26-b1c6-3d1c3b3c18f9 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:35.355753 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41b1d5-cc82-4263-8231-472275b55c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:35.356279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41b1d5-cc82-4263-8231-472275b55c4e None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:35.356556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41b1d5-cc82-4263-8231-472275b55c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:35.356825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41b1d5-cc82-4263-8231-472275b55c4e None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:35.357556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41b1d5-cc82-4263-8231-472275b55c4e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:35.357869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41b1d5-cc82-4263-8231-472275b55c4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:35.360028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41b1d5-cc82-4263-8231-472275b55c4e None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:35.384194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 354/700] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 16 05:24:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:35.389441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d6c57-c914-4c07-bcb6-ca8959b2011d None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:35.410448 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:35.504081 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d6c57-c914-4c07-bcb6-ca8959b2011d None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:35.546309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc942d-fc85-43e3-9200-60aad8f52e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:35.547130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc942d-fc85-43e3-9200-60aad8f52e6b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:35.547416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc942d-fc85-43e3-9200-60aad8f52e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:35.547683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc942d-fc85-43e3-9200-60aad8f52e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:35.584594 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41b1d5-cc82-4263-8231-472275b55c4e None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:35.601476 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:35.636705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc942d-fc85-43e3-9200-60aad8f52e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:35.638171 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc942d-fc85-43e3-9200-60aad8f52e6b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:35.660301 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:35.733095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41b1d5-cc82-4263-8231-472275b55c4e None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:35.770856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 347/701] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 16 05:24:35.776028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc942d-fc85-43e3-9200-60aad8f52e6b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:35.780102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c7c39-7604-4937-8970-5709287757a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:35.781030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c7c39-7604-4937-8970-5709287757a8 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:35.781435 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c7c39-7604-4937-8970-5709287757a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:35.781750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c7c39-7604-4937-8970-5709287757a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:35.795420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c7c39-7604-4937-8970-5709287757a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:35.797714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1c7c39-7604-4937-8970-5709287757a8 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-ad1c7c39-7604-4937-8970-5709287757a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:35.818626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:35.820869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:35.823066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:35.823066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:35.823066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:35.823066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 355/702] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:24:35.829868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c7c39-7604-4937-8970-5709287757a8 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:35.900724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc942d-fc85-43e3-9200-60aad8f52e6b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:35.921204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:35.927278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:35.936068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfefac6-8050-4b24-ba8a-bae3c0192bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:35.937260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfefac6-8050-4b24-ba8a-bae3c0192bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:35.937533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfefac6-8050-4b24-ba8a-bae3c0192bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:35.937796 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfefac6-8050-4b24-ba8a-bae3c0192bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:35.948951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfefac6-8050-4b24-ba8a-bae3c0192bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:35.950446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfefac6-8050-4b24-ba8a-bae3c0192bc5 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-2dfefac6-8050-4b24-ba8a-bae3c0192bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:35.969664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:35.973150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:35.974087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:35.974087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:35.974087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:35.974087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 348/703] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:24:35.979552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfefac6-8050-4b24-ba8a-bae3c0192bc5 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:35.982379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c7c39-7604-4937-8970-5709287757a8 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:36.028326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e19dd-2b69-4391-af8f-7a379ce3ca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:36.028933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e19dd-2b69-4391-af8f-7a379ce3ca99 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:36.029384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e19dd-2b69-4391-af8f-7a379ce3ca99 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:36.029661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e19dd-2b69-4391-af8f-7a379ce3ca99 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:36.031553 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e19dd-2b69-4391-af8f-7a379ce3ca99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:36.031869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e19dd-2b69-4391-af8f-7a379ce3ca99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:36.033247 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e19dd-2b69-4391-af8f-7a379ce3ca99 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:36.053604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 356/704] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:36.055312 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a2876-9a2c-47fb-a4af-2b8e4b054082 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:36.055600 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a2876-9a2c-47fb-a4af-2b8e4b054082 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:36.055870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a2876-9a2c-47fb-a4af-2b8e4b054082 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:36.058015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 357/705] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:36.058712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a2876-9a2c-47fb-a4af-2b8e4b054082 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:36.107902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfefac6-8050-4b24-ba8a-bae3c0192bc5 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:36.145188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd24724-a0b9-4969-9990-218a9c878c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:36.145792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd24724-a0b9-4969-9990-218a9c878c4a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:36.146092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd24724-a0b9-4969-9990-218a9c878c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:36.146401 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd24724-a0b9-4969-9990-218a9c878c4a None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:36.147037 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd24724-a0b9-4969-9990-218a9c878c4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:36.147333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd24724-a0b9-4969-9990-218a9c878c4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:36.149667 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd24724-a0b9-4969-9990-218a9c878c4a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:36.167685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 349/706] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:36.167685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cec37-e30a-467d-a949-0bc56da824bd None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:36.167685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cec37-e30a-467d-a949-0bc56da824bd None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:36.167685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cec37-e30a-467d-a949-0bc56da824bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:36.184808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-e15cec37-e30a-467d-a949-0bc56da824bd None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:36.184992 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a2876-9a2c-47fb-a4af-2b8e4b054082 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:36.186026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e15cec37-e30a-467d-a949-0bc56da824bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:36.210808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:36.212887 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:36.221189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66400b2-5835-42a0-bbed-542c28ddfa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:36.221793 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66400b2-5835-42a0-bbed-542c28ddfa1c None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:36.222096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66400b2-5835-42a0-bbed-542c28ddfa1c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:36.222363 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66400b2-5835-42a0-bbed-542c28ddfa1c None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:36.223060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66400b2-5835-42a0-bbed-542c28ddfa1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:36.224498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66400b2-5835-42a0-bbed-542c28ddfa1c None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:36.259440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 350/707] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:36.260652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954b777-eae3-4020-b04b-2a8cb7373611 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:36.260933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954b777-eae3-4020-b04b-2a8cb7373611 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:36.261197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8954b777-eae3-4020-b04b-2a8cb7373611 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:36.265866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 351/708] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:36.265866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954b777-eae3-4020-b04b-2a8cb7373611 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:36.309288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 358/709] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:24:36 2022] POST /identity/v3/users => generated 254 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:24:36.309288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d85e6d-e661-474c-bb7a-44bf886631de None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:36.309288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d85e6d-e661-474c-bb7a-44bf886631de None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:36.309288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d85e6d-e661-474c-bb7a-44bf886631de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:36.333481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c6d85e6d-e661-474c-bb7a-44bf886631de None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:36.335623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c6d85e6d-e661-474c-bb7a-44bf886631de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:36.422218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 359/710] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:36.445681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8954b777-eae3-4020-b04b-2a8cb7373611 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:36.495841 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34026a24-7862-4cdc-b86a-afb4949225c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:36.496481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34026a24-7862-4cdc-b86a-afb4949225c4 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:36.496753 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34026a24-7862-4cdc-b86a-afb4949225c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:36.497017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34026a24-7862-4cdc-b86a-afb4949225c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:36.504803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34026a24-7862-4cdc-b86a-afb4949225c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:36.506287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34026a24-7862-4cdc-b86a-afb4949225c4 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-34026a24-7862-4cdc-b86a-afb4949225c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:36.519567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:36.520559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:36.521484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:36.521484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:36.521484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:36.521484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:36.521484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:36.521484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:36.521484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:36.521484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:36.521484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 352/711] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:36.526509 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d85e6d-e661-474c-bb7a-44bf886631de None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:36.545420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34026a24-7862-4cdc-b86a-afb4949225c4 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:36.577671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:36.670187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d85e6d-e661-474c-bb7a-44bf886631de None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:36.684096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:36.695632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34026a24-7862-4cdc-b86a-afb4949225c4 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:36.710902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb299ff-a14d-41fe-87b1-01e645e0fcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:36.711842 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb299ff-a14d-41fe-87b1-01e645e0fcd2 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:36.712331 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb299ff-a14d-41fe-87b1-01e645e0fcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:36.712802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb299ff-a14d-41fe-87b1-01e645e0fcd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:36.713704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb299ff-a14d-41fe-87b1-01e645e0fcd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:36.714239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb299ff-a14d-41fe-87b1-01e645e0fcd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:36.717199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb299ff-a14d-41fe-87b1-01e645e0fcd2 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:36.731979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 360/712] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:36.738348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb299ff-a14d-41fe-87b1-01e645e0fcd2 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:36.744397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d16ddf-3ea4-4bf7-979b-764341bbb989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:36.745344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d16ddf-3ea4-4bf7-979b-764341bbb989 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:36.745969 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d16ddf-3ea4-4bf7-979b-764341bbb989 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:36.746444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d16ddf-3ea4-4bf7-979b-764341bbb989 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:36.747304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d16ddf-3ea4-4bf7-979b-764341bbb989 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:36.748916 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d16ddf-3ea4-4bf7-979b-764341bbb989 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-98d16ddf-3ea4-4bf7-979b-764341bbb989 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:36.769358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:36.770709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 05:24:36.771595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:36.773902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 353/713] 10.222.0.49 () {56 vars in 1129 bytes} [Fri Sep 16 05:24:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:24:36.774869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d16ddf-3ea4-4bf7-979b-764341bbb989 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:36.874768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb299ff-a14d-41fe-87b1-01e645e0fcd2 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:36.883546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:36.902427 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d16ddf-3ea4-4bf7-979b-764341bbb989 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:36.923764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c26aa-f00e-4fa7-8d09-9bd98e79c40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:36.924379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c26aa-f00e-4fa7-8d09-9bd98e79c40c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:36.924651 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c26aa-f00e-4fa7-8d09-9bd98e79c40c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:36.924913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c26aa-f00e-4fa7-8d09-9bd98e79c40c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:36.925639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c26aa-f00e-4fa7-8d09-9bd98e79c40c None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=7db75c20e9744feb8b5dc78c554f4977) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:36.925997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c26aa-f00e-4fa7-8d09-9bd98e79c40c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:36.927426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c26aa-f00e-4fa7-8d09-9bd98e79c40c None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:36.933732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 361/714] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:36 2022] GET /identity/v3/role_assignments?user.id=7db75c20e9744feb8b5dc78c554f4977&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:36.934668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c26aa-f00e-4fa7-8d09-9bd98e79c40c None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:36.952229 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2a786-b9ff-481d-b99e-222480f06c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:36.953267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2a786-b9ff-481d-b99e-222480f06c2e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:36.953777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2a786-b9ff-481d-b99e-222480f06c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:36.954296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2a786-b9ff-481d-b99e-222480f06c2e None admin] PATH_INFO: `/v3/services/network` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:36.955095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2a786-b9ff-481d-b99e-222480f06c2e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:36.956689 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2a786-b9ff-481d-b99e-222480f06c2e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-e9e2a786-b9ff-481d-b99e-222480f06c2e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:36.966620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:36.967791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 05:24:36.968657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:36.971658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 354/715] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:36 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:36.982223 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:36.984757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2a786-b9ff-481d-b99e-222480f06c2e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:37.066335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c26aa-f00e-4fa7-8d09-9bd98e79c40c None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:37.144778 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c037a-79b0-4c0f-9c56-2b2a4876a93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:37.145836 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c037a-79b0-4c0f-9c56-2b2a4876a93b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:37.146320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c037a-79b0-4c0f-9c56-2b2a4876a93b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:37.146833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c037a-79b0-4c0f-9c56-2b2a4876a93b None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:37.147659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c037a-79b0-4c0f-9c56-2b2a4876a93b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:37.154424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c037a-79b0-4c0f-9c56-2b2a4876a93b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:37.156893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c037a-79b0-4c0f-9c56-2b2a4876a93b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:37.165007 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 362/716] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Sep 16 05:24:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:37.173611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c037a-79b0-4c0f-9c56-2b2a4876a93b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:37.179209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2a786-b9ff-481d-b99e-222480f06c2e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:37.256969 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660ff66-24f5-40a9-b5e3-87ec41455c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:37.257956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660ff66-24f5-40a9-b5e3-87ec41455c89 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:37.258412 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660ff66-24f5-40a9-b5e3-87ec41455c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:37.258870 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2660ff66-24f5-40a9-b5e3-87ec41455c89 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:37.265846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660ff66-24f5-40a9-b5e3-87ec41455c89 None admin] RBAC: Adding query filter params (name=network) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:37.266443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660ff66-24f5-40a9-b5e3-87ec41455c89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:37.268299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2660ff66-24f5-40a9-b5e3-87ec41455c89 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:37.276161 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 355/717] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 16 05:24:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:37.292967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660ff66-24f5-40a9-b5e3-87ec41455c89 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:37.311264 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:37.315623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c037a-79b0-4c0f-9c56-2b2a4876a93b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:37.318947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:37.350975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6aea7-3584-4b23-9dbf-4af0a06ca550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:37.351911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6aea7-3584-4b23-9dbf-4af0a06ca550 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:37.352628 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6aea7-3584-4b23-9dbf-4af0a06ca550 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:37.353069 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6aea7-3584-4b23-9dbf-4af0a06ca550 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:37.354015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d6aea7-3584-4b23-9dbf-4af0a06ca550 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:37.354513 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d6aea7-3584-4b23-9dbf-4af0a06ca550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:37.356037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d6aea7-3584-4b23-9dbf-4af0a06ca550 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:37.369915 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 363/718] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Sep 16 05:24:37 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:37.489303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2660ff66-24f5-40a9-b5e3-87ec41455c89 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:37.527057 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bd076-05c5-4bb8-8e5b-c4f3e21a1367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:37.527596 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bd076-05c5-4bb8-8e5b-c4f3e21a1367 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:37.527872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bd076-05c5-4bb8-8e5b-c4f3e21a1367 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:37.528139 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bd076-05c5-4bb8-8e5b-c4f3e21a1367 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:37.528775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bd076-05c5-4bb8-8e5b-c4f3e21a1367 None admin] RBAC: Adding query filter params (type=network) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:37.529072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bd076-05c5-4bb8-8e5b-c4f3e21a1367 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:37.531168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bd076-05c5-4bb8-8e5b-c4f3e21a1367 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:37.537899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 356/719] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 16 05:24:37 2022] GET /identity/v3/services?type=network => generated 124 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:38.195652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a46e8e-be7a-40f1-b009-0c8b8ed5be2d None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:38.196155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a46e8e-be7a-40f1-b009-0c8b8ed5be2d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:38.196155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a46e8e-be7a-40f1-b009-0c8b8ed5be2d None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:38.197708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 364/720] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:38.203377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f757e-9f73-4e6a-b196-8f77b628daf2 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:38.203771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f757e-9f73-4e6a-b196-8f77b628daf2 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:38.203937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f757e-9f73-4e6a-b196-8f77b628daf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:38.222858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-485f757e-9f73-4e6a-b196-8f77b628daf2 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:38.224024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-485f757e-9f73-4e6a-b196-8f77b628daf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:38.311132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 357/721] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:38.665851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a46e8e-be7a-40f1-b009-0c8b8ed5be2d None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:38.805543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a46e8e-be7a-40f1-b009-0c8b8ed5be2d None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:38.865235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72bb71-e992-4e27-b262-3635e9e7b4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:38.865880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72bb71-e992-4e27-b262-3635e9e7b4ba None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:38.866152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72bb71-e992-4e27-b262-3635e9e7b4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:38.866415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72bb71-e992-4e27-b262-3635e9e7b4ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:38.878714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72bb71-e992-4e27-b262-3635e9e7b4ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-2e72bb71-e992-4e27-b262-3635e9e7b4ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:38.884083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:38.885006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:38.885889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:38.885889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:38.885889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:38.885889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:38.885889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:38.885889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:38.885889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:38.885889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:38.885889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:38.885889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:38.885889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:38.885889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:38.885889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 365/722] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:24:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:38.892171 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f757e-9f73-4e6a-b196-8f77b628daf2 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:38.903247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:39.054157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f757e-9f73-4e6a-b196-8f77b628daf2 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:39.101122 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e21f5-172e-45eb-a82a-8eb2c06bcf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:39.101751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e21f5-172e-45eb-a82a-8eb2c06bcf9e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:39.102061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e21f5-172e-45eb-a82a-8eb2c06bcf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:39.102330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e21f5-172e-45eb-a82a-8eb2c06bcf9e None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:39.103006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e21f5-172e-45eb-a82a-8eb2c06bcf9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:39.103304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e21f5-172e-45eb-a82a-8eb2c06bcf9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:39.104684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e21f5-172e-45eb-a82a-8eb2c06bcf9e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:39.122069 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 358/723] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:39.127908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72bb71-e992-4e27-b262-3635e9e7b4ba None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:39.167042 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:39.173421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b2186-3201-45c2-8ee5-15859b57371f None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:39.173681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b2186-3201-45c2-8ee5-15859b57371f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:39.173946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b2186-3201-45c2-8ee5-15859b57371f None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:39.175454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 359/724] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:39.206035 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f1c9b-786f-40a7-b754-589006e943c4 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:39.206662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f1c9b-786f-40a7-b754-589006e943c4 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:39.206662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f1c9b-786f-40a7-b754-589006e943c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:39.245061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-e62f1c9b-786f-40a7-b754-589006e943c4 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:39.246290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e62f1c9b-786f-40a7-b754-589006e943c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:39.300304 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72bb71-e992-4e27-b262-3635e9e7b4ba None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:39.370199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:39.370695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d3b14-aac3-465a-9280-8618a6d283a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:39.371328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d3b14-aac3-465a-9280-8618a6d283a4 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:39.371609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d3b14-aac3-465a-9280-8618a6d283a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:39.371888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d3b14-aac3-465a-9280-8618a6d283a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:39.384450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 360/725] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 16 05:24:39.388249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d3b14-aac3-465a-9280-8618a6d283a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:39.390108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d3b14-aac3-465a-9280-8618a6d283a4 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-b95d3b14-aac3-465a-9280-8618a6d283a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:39.405471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:39.406542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:39.407491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:39.407491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:39.407491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:39.407491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 366/726] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:39.416134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f1c9b-786f-40a7-b754-589006e943c4 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:39.549531 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f1c9b-786f-40a7-b754-589006e943c4 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:39.596259 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcda62-471f-43a4-bacf-613da8c2daaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:39.597243 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcda62-471f-43a4-bacf-613da8c2daaf None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:39.597726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcda62-471f-43a4-bacf-613da8c2daaf None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:39.598253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcda62-471f-43a4-bacf-613da8c2daaf None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:39.599263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcda62-471f-43a4-bacf-613da8c2daaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:39.599766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcda62-471f-43a4-bacf-613da8c2daaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:39.601345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbcda62-471f-43a4-bacf-613da8c2daaf None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:39.618850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 361/727] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:39.625086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d3b14-aac3-465a-9280-8618a6d283a4 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:39.730984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:39.751679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcda62-471f-43a4-bacf-613da8c2daaf None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:39.771384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d3b14-aac3-465a-9280-8618a6d283a4 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:39.841306 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e9679-8e8b-4974-ad2e-882432e785d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:39.841874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e9679-8e8b-4974-ad2e-882432e785d7 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:39.842150 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e9679-8e8b-4974-ad2e-882432e785d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:39.842414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e9679-8e8b-4974-ad2e-882432e785d7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:39.853157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e9679-8e8b-4974-ad2e-882432e785d7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:39.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e9679-8e8b-4974-ad2e-882432e785d7 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:39.865492 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-372e9679-8e8b-4974-ad2e-882432e785d7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:39.875527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:39.879442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:39.880420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:39.880420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:39.880420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:39.880420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:39.880420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:39.880420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:39.880420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:39.880420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:39.880420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 05:24:39.880420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:39.881393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 367/728] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:24:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:39.882680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ea794-653f-421b-b078-4253fa45a0fe None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:39.882958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ea794-653f-421b-b078-4253fa45a0fe None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:39.883224 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ea794-653f-421b-b078-4253fa45a0fe None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:39.884961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 368/729] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:39.885962 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ea794-653f-421b-b078-4253fa45a0fe None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:39.919946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcda62-471f-43a4-bacf-613da8c2daaf None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:39.957342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548342b9-fe3c-41f3-9cb9-03f9e31c44cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:39.958015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548342b9-fe3c-41f3-9cb9-03f9e31c44cc None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:39.958294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548342b9-fe3c-41f3-9cb9-03f9e31c44cc None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:39.958561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548342b9-fe3c-41f3-9cb9-03f9e31c44cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:39.970130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548342b9-fe3c-41f3-9cb9-03f9e31c44cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-548342b9-fe3c-41f3-9cb9-03f9e31c44cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:39.975583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:39.976726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:39.977881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:39.977881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:39.977881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:39.977881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:39.977881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:39.977881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:39.977881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:39.977881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:39.977881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:39.977881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:39.977881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:39.977881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:39.981701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 362/730] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:39.983082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62683db5-45fb-44b6-abcb-b73b2429e7be None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:39.983911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62683db5-45fb-44b6-abcb-b73b2429e7be None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:39.984186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62683db5-45fb-44b6-abcb-b73b2429e7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:40.004539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-62683db5-45fb-44b6-abcb-b73b2429e7be None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:40.005779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-62683db5-45fb-44b6-abcb-b73b2429e7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:40.015136 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ea794-653f-421b-b078-4253fa45a0fe None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:40.058213 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53852c-dc55-4aed-b645-f4242632489f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:40.059073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53852c-dc55-4aed-b645-f4242632489f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:40.059525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53852c-dc55-4aed-b645-f4242632489f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:40.059894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53852c-dc55-4aed-b645-f4242632489f None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:40.060776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53852c-dc55-4aed-b645-f4242632489f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:40.061318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53852c-dc55-4aed-b645-f4242632489f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:40.063064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53852c-dc55-4aed-b645-f4242632489f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:40.065036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:40.082221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 363/731] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:40.083296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62683db5-45fb-44b6-abcb-b73b2429e7be None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:40.083781 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 369/732] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Sep 16 05:24:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:40.095331 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53852c-dc55-4aed-b645-f4242632489f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:40.148526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:40.200224 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62683db5-45fb-44b6-abcb-b73b2429e7be None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:40.219385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53852c-dc55-4aed-b645-f4242632489f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:40.237245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc109a5f-362f-4e7b-81e6-99c743f17093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:40.238499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc109a5f-362f-4e7b-81e6-99c743f17093 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:40.239005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc109a5f-362f-4e7b-81e6-99c743f17093 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:40.239477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc109a5f-362f-4e7b-81e6-99c743f17093 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:40.240346 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc109a5f-362f-4e7b-81e6-99c743f17093 None admin] RBAC: Adding query filter params (name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:40.240901 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc109a5f-362f-4e7b-81e6-99c743f17093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:40.242555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc109a5f-362f-4e7b-81e6-99c743f17093 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:40.256599 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 364/733] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:40.256599 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb813c-5549-443e-ac79-ef71b5cf0a8e None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:40.256599 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb813c-5549-443e-ac79-ef71b5cf0a8e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:40.256919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8594b7f-9366-400d-947f-b84286f58b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:40.257014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb813c-5549-443e-ac79-ef71b5cf0a8e None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:40.257088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8594b7f-9366-400d-947f-b84286f58b5c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:40.257088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8594b7f-9366-400d-947f-b84286f58b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:40.257088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8594b7f-9366-400d-947f-b84286f58b5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:40.257446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 365/734] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:40.258530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cbfa5-68c6-44d3-94b8-44c072e358e7 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:40.258809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cbfa5-68c6-44d3-94b8-44c072e358e7 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:40.259080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cbfa5-68c6-44d3-94b8-44c072e358e7 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:40.261949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 366/735] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:40.263675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cbfa5-68c6-44d3-94b8-44c072e358e7 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:40.323638 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:40.334401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8594b7f-9366-400d-947f-b84286f58b5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:40.336864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8594b7f-9366-400d-947f-b84286f58b5c None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:40.373187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5cbfa5-68c6-44d3-94b8-44c072e358e7 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:40.383818 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:40.408696 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00743d7c-fa72-4304-b737-365447f3cc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:40.409320 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00743d7c-fa72-4304-b737-365447f3cc3a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:40.409596 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00743d7c-fa72-4304-b737-365447f3cc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:40.409896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00743d7c-fa72-4304-b737-365447f3cc3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:40.420691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00743d7c-fa72-4304-b737-365447f3cc3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:40.425848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00743d7c-fa72-4304-b737-365447f3cc3a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-00743d7c-fa72-4304-b737-365447f3cc3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:40.440866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:40.441913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:40.442903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:40.442903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:40.442903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:40.442903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 367/736] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:40.443137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c00420-76dc-49c9-b8da-b703e6c39b9c None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:40.443482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c00420-76dc-49c9-b8da-b703e6c39b9c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:40.443662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c00420-76dc-49c9-b8da-b703e6c39b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:40.443759 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 370/737] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 16 05:24:40.445105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7832c7f-4e06-4b28-9aad-b401843fe2f1 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:40.445387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7832c7f-4e06-4b28-9aad-b401843fe2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:40.445656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7832c7f-4e06-4b28-9aad-b401843fe2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:40.466734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-96c00420-76dc-49c9-b8da-b703e6c39b9c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:40.468163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-96c00420-76dc-49c9-b8da-b703e6c39b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:40.470411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-f7832c7f-4e06-4b28-9aad-b401843fe2f1 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:40.471540 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f7832c7f-4e06-4b28-9aad-b401843fe2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:40.534313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 368/738] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:40.535404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c00420-76dc-49c9-b8da-b703e6c39b9c None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:40.536866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 371/739] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 16 05:24:40.537680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7832c7f-4e06-4b28-9aad-b401843fe2f1 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:40.570119 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:40.651043 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:40.660603 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c00420-76dc-49c9-b8da-b703e6c39b9c None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:40.667232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7832c7f-4e06-4b28-9aad-b401843fe2f1 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:40.701919 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fdc39-ae0f-44a3-b122-81e14d9f5052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:40.703595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fdc39-ae0f-44a3-b122-81e14d9f5052 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:40.704057 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fdc39-ae0f-44a3-b122-81e14d9f5052 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:40.704534 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fdc39-ae0f-44a3-b122-81e14d9f5052 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:40.707233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ba472-2b8b-4fb2-bab5-1e1442dddafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:40.708042 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ba472-2b8b-4fb2-bab5-1e1442dddafc None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:40.708377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ba472-2b8b-4fb2-bab5-1e1442dddafc None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:40.708716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ba472-2b8b-4fb2-bab5-1e1442dddafc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:40.718758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fdc39-ae0f-44a3-b122-81e14d9f5052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:40.720443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ba472-2b8b-4fb2-bab5-1e1442dddafc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:40.722083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ba472-2b8b-4fb2-bab5-1e1442dddafc None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-e96fdc39-ae0f-44a3-b122-81e14d9f5052 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:40.725622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:40.726631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:40.727487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:40.727487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:40.727487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:40.727487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:40.727487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:40.727487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:40.727487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:40.727487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:40.727487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:40.727487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:40.727487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:40.727487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:40.731966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 369/740] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:24:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:40.735185 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fdc39-ae0f-44a3-b122-81e14d9f5052 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-e35ba472-2b8b-4fb2-bab5-1e1442dddafc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:40.740238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:40.741196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:40.742139 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:40.742139 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:40.742139 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:40.746278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 372/741] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:40.747149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ba472-2b8b-4fb2-bab5-1e1442dddafc None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:40.831037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:40.855417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ba472-2b8b-4fb2-bab5-1e1442dddafc None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:40.855581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fdc39-ae0f-44a3-b122-81e14d9f5052 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:40.890182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8a54a-fa2b-4b6d-820c-642d12fee3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:40.890725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14194b4-186e-4a03-bdff-9b27d802eb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:40.890891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8a54a-fa2b-4b6d-820c-642d12fee3fb None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:40.891167 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8a54a-fa2b-4b6d-820c-642d12fee3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:40.891322 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14194b4-186e-4a03-bdff-9b27d802eb08 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:40.891423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8a54a-fa2b-4b6d-820c-642d12fee3fb None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:40.891851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14194b4-186e-4a03-bdff-9b27d802eb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:40.892119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14194b4-186e-4a03-bdff-9b27d802eb08 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:40.892333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a8a54a-fa2b-4b6d-820c-642d12fee3fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:40.892633 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a8a54a-fa2b-4b6d-820c-642d12fee3fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:40.892750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14194b4-186e-4a03-bdff-9b27d802eb08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:40.893031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14194b4-186e-4a03-bdff-9b27d802eb08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:40.894004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a8a54a-fa2b-4b6d-820c-642d12fee3fb None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:40.894408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14194b4-186e-4a03-bdff-9b27d802eb08 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:40.901875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 373/742] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:40.902596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8a54a-fa2b-4b6d-820c-642d12fee3fb None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:40.909580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 370/743] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:40.913049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14194b4-186e-4a03-bdff-9b27d802eb08 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:40.926070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:41.008602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8a54a-fa2b-4b6d-820c-642d12fee3fb None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:41.014681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14194b4-186e-4a03-bdff-9b27d802eb08 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:41.041398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda29f59-0c34-45db-9d73-e0086621a7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:41.042029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda29f59-0c34-45db-9d73-e0086621a7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:41.042307 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda29f59-0c34-45db-9d73-e0086621a7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:41.042582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda29f59-0c34-45db-9d73-e0086621a7a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:41.043210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda29f59-0c34-45db-9d73-e0086621a7a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:41.043507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda29f59-0c34-45db-9d73-e0086621a7a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:41.044852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda29f59-0c34-45db-9d73-e0086621a7a0 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:41.047742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:41.049092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e31d7-3af5-4cc9-bbb8-038cfc173498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:41.049684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e31d7-3af5-4cc9-bbb8-038cfc173498 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:41.049996 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e31d7-3af5-4cc9-bbb8-038cfc173498 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:41.050264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e31d7-3af5-4cc9-bbb8-038cfc173498 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:41.050903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e31d7-3af5-4cc9-bbb8-038cfc173498 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:41.052233 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e31d7-3af5-4cc9-bbb8-038cfc173498 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:41.057252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 374/744] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:41.058059 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda29f59-0c34-45db-9d73-e0086621a7a0 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:41.132535 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 371/745] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:24:40 2022] POST /identity/v3/services => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:24:41.133519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e31d7-3af5-4cc9-bbb8-038cfc173498 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:41.158506 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda29f59-0c34-45db-9d73-e0086621a7a0 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:41.203661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a2375-7241-46ef-a67b-39778233e970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:41.204552 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a2375-7241-46ef-a67b-39778233e970 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:41.204834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a2375-7241-46ef-a67b-39778233e970 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:41.205100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607a2375-7241-46ef-a67b-39778233e970 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:41.205816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a2375-7241-46ef-a67b-39778233e970 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:41.207266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607a2375-7241-46ef-a67b-39778233e970 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:41.221872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:41.256252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e31d7-3af5-4cc9-bbb8-038cfc173498 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:41.277905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 375/746] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:24:41 2022] POST /identity/v3/services => generated 240 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:24:41.277905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a2375-7241-46ef-a67b-39778233e970 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:41.321732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de7847-45f9-4aef-849b-718b5f9c8e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:41.323992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de7847-45f9-4aef-849b-718b5f9c8e04 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:41.325566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de7847-45f9-4aef-849b-718b5f9c8e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:41.326650 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de7847-45f9-4aef-849b-718b5f9c8e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:41.343959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de7847-45f9-4aef-849b-718b5f9c8e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:41.345449 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de7847-45f9-4aef-849b-718b5f9c8e04 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:41.356018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-f1de7847-45f9-4aef-849b-718b5f9c8e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:41.363551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:41.367747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:41.368888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:41.368888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:41.368888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:41.368888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 372/747] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:41.368888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de7847-45f9-4aef-849b-718b5f9c8e04 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:41.427027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607a2375-7241-46ef-a67b-39778233e970 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:41.476375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d3f5d-d00e-4749-b4b2-a4cb9a2e1bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:41.476584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d3f5d-d00e-4749-b4b2-a4cb9a2e1bfc None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:41.476792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d3f5d-d00e-4749-b4b2-a4cb9a2e1bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:41.477036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d3f5d-d00e-4749-b4b2-a4cb9a2e1bfc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:41.488785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d3f5d-d00e-4749-b4b2-a4cb9a2e1bfc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:41.490292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d3f5d-d00e-4749-b4b2-a4cb9a2e1bfc None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-a00d3f5d-d00e-4749-b4b2-a4cb9a2e1bfc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:41.504907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:41.506187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:41.507143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:41.507143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:41.507143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:41.507143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:41.507143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:41.507143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:41.507143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:41.507143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:41.507143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 05:24:41.507143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:41.509912 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 376/748] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 16 05:24:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:41.511331 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d3f5d-d00e-4749-b4b2-a4cb9a2e1bfc None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:41.518766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de7847-45f9-4aef-849b-718b5f9c8e04 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:41.540427 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:41.563076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec42946-9e0b-4518-a62f-6795865d2dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:41.563703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec42946-9e0b-4518-a62f-6795865d2dbe None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:41.563974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec42946-9e0b-4518-a62f-6795865d2dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:41.564237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec42946-9e0b-4518-a62f-6795865d2dbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:41.572249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec42946-9e0b-4518-a62f-6795865d2dbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:41.573728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec42946-9e0b-4518-a62f-6795865d2dbe None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-2ec42946-9e0b-4518-a62f-6795865d2dbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:41.586832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:41.587775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:41.589086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:41.589086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:41.589086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:41.589086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:41.589086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:41.589086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:41.589086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:41.589086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:41.589086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 373/749] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:41.589574 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec42946-9e0b-4518-a62f-6795865d2dbe None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:41.642956 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d3f5d-d00e-4749-b4b2-a4cb9a2e1bfc None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:41.679951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:41.709559 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ede14-422a-4bb6-a71c-25861d5817c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:41.710522 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ede14-422a-4bb6-a71c-25861d5817c5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:41.710969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ede14-422a-4bb6-a71c-25861d5817c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:41.711403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ede14-422a-4bb6-a71c-25861d5817c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:41.712213 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ede14-422a-4bb6-a71c-25861d5817c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:41.712682 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ede14-422a-4bb6-a71c-25861d5817c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:41.713575 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec42946-9e0b-4518-a62f-6795865d2dbe None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:41.714452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ede14-422a-4bb6-a71c-25861d5817c5 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:41.731314 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 377/750] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:41.732479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ede14-422a-4bb6-a71c-25861d5817c5 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:41.752614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dc221-4e98-438a-ab0a-3f8e553668da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:41.753307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc221-4e98-438a-ab0a-3f8e553668da None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:41.753705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc221-4e98-438a-ab0a-3f8e553668da None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:41.754083 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dc221-4e98-438a-ab0a-3f8e553668da None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:41.754874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dc221-4e98-438a-ab0a-3f8e553668da None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:41.755241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dc221-4e98-438a-ab0a-3f8e553668da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:41.756930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dc221-4e98-438a-ab0a-3f8e553668da None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:41.778419 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 374/751] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Sep 16 05:24:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:41.778419 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dc221-4e98-438a-ab0a-3f8e553668da None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:41.816946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:41.843172 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ede14-422a-4bb6-a71c-25861d5817c5 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:41.878849 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63380d77-6eff-4b87-a34d-2767563caa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:41.879923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63380d77-6eff-4b87-a34d-2767563caa58 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:41.880396 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63380d77-6eff-4b87-a34d-2767563caa58 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:41.880894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63380d77-6eff-4b87-a34d-2767563caa58 None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:41.881791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63380d77-6eff-4b87-a34d-2767563caa58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:41.882308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63380d77-6eff-4b87-a34d-2767563caa58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:41.883925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63380d77-6eff-4b87-a34d-2767563caa58 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:41.890547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dc221-4e98-438a-ab0a-3f8e553668da None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:41.902200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 378/752] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:41.903370 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63380d77-6eff-4b87-a34d-2767563caa58 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:41.926899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee9512-7661-499a-b00b-944b35117a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:41.928017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee9512-7661-499a-b00b-944b35117a8a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:41.928017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee9512-7661-499a-b00b-944b35117a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:41.928154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee9512-7661-499a-b00b-944b35117a8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:41.935827 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee9512-7661-499a-b00b-944b35117a8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:41.937924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee9512-7661-499a-b00b-944b35117a8a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:41.948073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-03ee9512-7661-499a-b00b-944b35117a8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:41.950188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:41.951140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:41.952820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:41.952820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:41.952820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:41.952820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:41.952820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:41.952820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:41.952820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:41.952820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:41.952820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:24:41.952820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:41.956128 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 375/753] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 16 05:24:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:41.957732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee9512-7661-499a-b00b-944b35117a8a None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:42.025206 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63380d77-6eff-4b87-a34d-2767563caa58 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:42.057250 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:42.061211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492455c-dbf3-405f-b891-18f762ff45bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:42.062215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492455c-dbf3-405f-b891-18f762ff45bb None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:42.062692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492455c-dbf3-405f-b891-18f762ff45bb None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:42.063179 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492455c-dbf3-405f-b891-18f762ff45bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:42.064085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee9512-7661-499a-b00b-944b35117a8a None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:42.100606 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a151e-ce59-4e86-88ed-ccfad01f2670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:42.101228 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a151e-ce59-4e86-88ed-ccfad01f2670 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:42.101509 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a151e-ce59-4e86-88ed-ccfad01f2670 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:42.101791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a151e-ce59-4e86-88ed-ccfad01f2670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:42.102556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a151e-ce59-4e86-88ed-ccfad01f2670 None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=b260fed0d99a4d3fa8f9c2ad79223219) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:42.102863 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a151e-ce59-4e86-88ed-ccfad01f2670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:42.104342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a151e-ce59-4e86-88ed-ccfad01f2670 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:42.116043 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 376/754] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:41 2022] GET /identity/v3/role_assignments?user.id=b260fed0d99a4d3fa8f9c2ad79223219&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:42.116043 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a151e-ce59-4e86-88ed-ccfad01f2670 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:42.169759 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492455c-dbf3-405f-b891-18f762ff45bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:42.171716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492455c-dbf3-405f-b891-18f762ff45bb None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:42.220181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a151e-ce59-4e86-88ed-ccfad01f2670 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:42.230170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:42.257874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e213e5f-f4af-4fca-9b05-0acf0caec3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:42.257874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e213e5f-f4af-4fca-9b05-0acf0caec3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:42.257874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e213e5f-f4af-4fca-9b05-0acf0caec3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:42.257874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e213e5f-f4af-4fca-9b05-0acf0caec3c3 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:42.258622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e213e5f-f4af-4fca-9b05-0acf0caec3c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:42.258901 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e213e5f-f4af-4fca-9b05-0acf0caec3c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:42.260997 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e213e5f-f4af-4fca-9b05-0acf0caec3c3 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:42.292408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 377/755] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 16 05:24:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:42.299498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e213e5f-f4af-4fca-9b05-0acf0caec3c3 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:42.320526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:42.414355 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e213e5f-f4af-4fca-9b05-0acf0caec3c3 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:42.421383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 379/756] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:24:42.430834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492455c-dbf3-405f-b891-18f762ff45bb None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:42.455991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9af0-b1ce-482e-8d8e-298ef595d5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:42.456624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9af0-b1ce-482e-8d8e-298ef595d5ef None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:42.456902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9af0-b1ce-482e-8d8e-298ef595d5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:42.457156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e9af0-b1ce-482e-8d8e-298ef595d5ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:42.468120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e9af0-b1ce-482e-8d8e-298ef595d5ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:42.470328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e9af0-b1ce-482e-8d8e-298ef595d5ef None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-652e9af0-b1ce-482e-8d8e-298ef595d5ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:42.491258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:42.495379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:42.496270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:42.496270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:42.496270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:42.496270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 378/757] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:42.498754 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9af0-b1ce-482e-8d8e-298ef595d5ef None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:42.580200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:42.619222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e9af0-b1ce-482e-8d8e-298ef595d5ef None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:42.646954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492455c-dbf3-405f-b891-18f762ff45bb None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:42.668422 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d597eb5-39ec-43ff-8b3d-00b82d7cc06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:42.669036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d597eb5-39ec-43ff-8b3d-00b82d7cc06a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:42.669316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d597eb5-39ec-43ff-8b3d-00b82d7cc06a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:42.669588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d597eb5-39ec-43ff-8b3d-00b82d7cc06a None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:42.670265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d597eb5-39ec-43ff-8b3d-00b82d7cc06a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:42.670567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d597eb5-39ec-43ff-8b3d-00b82d7cc06a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:42.671981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d597eb5-39ec-43ff-8b3d-00b82d7cc06a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:42.685035 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 379/758] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:42.690743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d597eb5-39ec-43ff-8b3d-00b82d7cc06a None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:42.764988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:42.787378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ec5b0-b278-4deb-9b2a-50e63865bed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:42.788040 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ec5b0-b278-4deb-9b2a-50e63865bed1 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:42.788313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ec5b0-b278-4deb-9b2a-50e63865bed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:42.788585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ec5b0-b278-4deb-9b2a-50e63865bed1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:42.808249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d597eb5-39ec-43ff-8b3d-00b82d7cc06a None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:42.811720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ec5b0-b278-4deb-9b2a-50e63865bed1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:42.821884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ec5b0-b278-4deb-9b2a-50e63865bed1 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-361ec5b0-b278-4deb-9b2a-50e63865bed1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:42.843909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:42.845100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:42.846235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:42.846235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:42.846235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:42.851336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 380/759] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:42.859400 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ec5b0-b278-4deb-9b2a-50e63865bed1 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:42.863501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:42.866996 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ab8cc-ddba-4ec2-9331-6a18bbc63be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:42.867798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ab8cc-ddba-4ec2-9331-6a18bbc63be8 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:42.868247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ab8cc-ddba-4ec2-9331-6a18bbc63be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:42.868662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549ab8cc-ddba-4ec2-9331-6a18bbc63be8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:42.875816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ab8cc-ddba-4ec2-9331-6a18bbc63be8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:42.877412 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549ab8cc-ddba-4ec2-9331-6a18bbc63be8 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-549ab8cc-ddba-4ec2-9331-6a18bbc63be8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:42.889008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:42.890375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:42.891505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:42.891505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:42.891505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:42.891505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:42.891505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:42.891505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:42.891505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:42.891505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:42.893786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 380/760] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:42.907064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ab8cc-ddba-4ec2-9331-6a18bbc63be8 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:43.023771 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ec5b0-b278-4deb-9b2a-50e63865bed1 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:43.071751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549ab8cc-ddba-4ec2-9331-6a18bbc63be8 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:43.100322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e9d48-af6a-47d5-abf1-735124b1fb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:43.100940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e9d48-af6a-47d5-abf1-735124b1fb44 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:43.101212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e9d48-af6a-47d5-abf1-735124b1fb44 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:43.101483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e9d48-af6a-47d5-abf1-735124b1fb44 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:43.102283 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e9d48-af6a-47d5-abf1-735124b1fb44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:43.102594 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e9d48-af6a-47d5-abf1-735124b1fb44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:43.104887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e9d48-af6a-47d5-abf1-735124b1fb44 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:43.128770 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 381/761] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:43.132563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e9d48-af6a-47d5-abf1-735124b1fb44 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:43.143518 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19171453-728e-48eb-ad73-9def06f166e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:43.144187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19171453-728e-48eb-ad73-9def06f166e8 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:43.144462 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19171453-728e-48eb-ad73-9def06f166e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:43.144728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19171453-728e-48eb-ad73-9def06f166e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:43.145461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19171453-728e-48eb-ad73-9def06f166e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:43.145765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19171453-728e-48eb-ad73-9def06f166e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:43.147213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19171453-728e-48eb-ad73-9def06f166e8 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:43.169337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 381/762] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:43.177456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19171453-728e-48eb-ad73-9def06f166e8 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:43.214712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:43.275026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e9d48-af6a-47d5-abf1-735124b1fb44 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:43.315265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1ca6f-57c0-4520-846a-c57def1e58f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:43.316398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1ca6f-57c0-4520-846a-c57def1e58f0 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:43.316881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1ca6f-57c0-4520-846a-c57def1e58f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:43.317354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1ca6f-57c0-4520-846a-c57def1e58f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:43.325652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1ca6f-57c0-4520-846a-c57def1e58f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:43.327598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1ca6f-57c0-4520-846a-c57def1e58f0 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-b9d1ca6f-57c0-4520-846a-c57def1e58f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:43.340548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:43.341547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:43.342487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:43.342487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:43.342487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:43.342487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:43.342487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:43.342487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:43.342487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:43.342487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:43.345374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 382/763] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:43.351424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1ca6f-57c0-4520-846a-c57def1e58f0 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:43.435730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:43.509988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1ca6f-57c0-4520-846a-c57def1e58f0 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:43.514008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19171453-728e-48eb-ad73-9def06f166e8 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:43.516968 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:43.566803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78744c-a74d-4e13-bd5a-01496444030b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:43.567339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78744c-a74d-4e13-bd5a-01496444030b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:43.567611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78744c-a74d-4e13-bd5a-01496444030b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:43.567908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78744c-a74d-4e13-bd5a-01496444030b None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:43.568941 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78744c-a74d-4e13-bd5a-01496444030b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:43.569245 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78744c-a74d-4e13-bd5a-01496444030b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:43.571381 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78744c-a74d-4e13-bd5a-01496444030b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:43.587318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 383/764] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:43.596453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78744c-a74d-4e13-bd5a-01496444030b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:43.611848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0168cf2-2498-4a65-a06f-e8724a8191d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:43.612868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0168cf2-2498-4a65-a06f-e8724a8191d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:43.613345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0168cf2-2498-4a65-a06f-e8724a8191d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:43.613814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0168cf2-2498-4a65-a06f-e8724a8191d8 None admin] PATH_INFO: `/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/7db75c20e9744feb8b5dc78c554f4977/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:43.616071 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0168cf2-2498-4a65-a06f-e8724a8191d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=29b2e27932984de79ee8dfbe95312ac2, user_id=7db75c20e9744feb8b5dc78c554f4977, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:43.617711 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0168cf2-2498-4a65-a06f-e8724a8191d8 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:43.716500 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 382/765] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:24:43 2022] PUT /identity/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/7db75c20e9744feb8b5dc78c554f4977/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:24:43.723280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78744c-a74d-4e13-bd5a-01496444030b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:43.726818 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d552c1f-ff23-4c96-8e3f-3c6291fa7029 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:43.727208 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d552c1f-ff23-4c96-8e3f-3c6291fa7029 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:43.727483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d552c1f-ff23-4c96-8e3f-3c6291fa7029 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:43.729323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 383/766] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:43.735381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b47c23-840c-48e2-91e2-a6fc1ee97d4e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:43.735685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b47c23-840c-48e2-91e2-a6fc1ee97d4e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:43.735915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b47c23-840c-48e2-91e2-a6fc1ee97d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:43.770326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a353490-91e5-4dd5-ba77-7622aa9d0935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:43.770944 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a353490-91e5-4dd5-ba77-7622aa9d0935 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:43.771220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a353490-91e5-4dd5-ba77-7622aa9d0935 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:43.771536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a353490-91e5-4dd5-ba77-7622aa9d0935 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:43.772273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a353490-91e5-4dd5-ba77-7622aa9d0935 None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=ba7a3df6fb6a4eb3a077a21b40794754) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:43.772581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a353490-91e5-4dd5-ba77-7622aa9d0935 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:43.774211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a353490-91e5-4dd5-ba77-7622aa9d0935 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:43.775057 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-10b47c23-840c-48e2-91e2-a6fc1ee97d4e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:43.776387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-10b47c23-840c-48e2-91e2-a6fc1ee97d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:43.781490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 384/767] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:43 2022] GET /identity/v3/role_assignments?user.id=ba7a3df6fb6a4eb3a077a21b40794754&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:43.841528 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe673fb6-f580-4b6f-a483-dd579e501f54 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:43.841724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe673fb6-f580-4b6f-a483-dd579e501f54 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:43.842022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe673fb6-f580-4b6f-a483-dd579e501f54 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:43.844622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 385/768] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:24:43.853688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da37e03-8d0f-4be3-b4b3-871bc2cdde70 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:43.854013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da37e03-8d0f-4be3-b4b3-871bc2cdde70 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:43.854293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da37e03-8d0f-4be3-b4b3-871bc2cdde70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:43.884503 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 384/769] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:24:43.893497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-5da37e03-8d0f-4be3-b4b3-871bc2cdde70 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:43.894990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5da37e03-8d0f-4be3-b4b3-871bc2cdde70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:43.935504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:43.994839 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 386/770] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:24:44.223255 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b47c23-840c-48e2-91e2-a6fc1ee97d4e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:44.267985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:44.345168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b47c23-840c-48e2-91e2-a6fc1ee97d4e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:44.389584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0f6e9-79f9-4c3a-9030-b7e8da3e6fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:44.390708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0f6e9-79f9-4c3a-9030-b7e8da3e6fbc None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:44.391190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0f6e9-79f9-4c3a-9030-b7e8da3e6fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:44.391922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0f6e9-79f9-4c3a-9030-b7e8da3e6fbc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:44.392707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0f6e9-79f9-4c3a-9030-b7e8da3e6fbc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:44.398406 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0f6e9-79f9-4c3a-9030-b7e8da3e6fbc None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-5db0f6e9-79f9-4c3a-9030-b7e8da3e6fbc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:44.409119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:44.410132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 05:24:44.410986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:44.413376 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 385/771] 10.222.0.49 () {56 vars in 1129 bytes} [Fri Sep 16 05:24:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:44.420201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da37e03-8d0f-4be3-b4b3-871bc2cdde70 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:44.423450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0f6e9-79f9-4c3a-9030-b7e8da3e6fbc None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:44.544097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0f6e9-79f9-4c3a-9030-b7e8da3e6fbc None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:44.546930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:44.583035 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4baf6-69c3-4f06-b90f-4437dbd0b4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:44.583667 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4baf6-69c3-4f06-b90f-4437dbd0b4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:44.584197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4baf6-69c3-4f06-b90f-4437dbd0b4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:44.584469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4baf6-69c3-4f06-b90f-4437dbd0b4e4 None admin] PATH_INFO: `/v3/services/network` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:44.585047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4baf6-69c3-4f06-b90f-4437dbd0b4e4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:44.588044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4baf6-69c3-4f06-b90f-4437dbd0b4e4 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-fbb4baf6-69c3-4f06-b90f-4437dbd0b4e4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:44.603657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:44.604702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:44.606220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 386/772] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:44 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:24:44.610909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4baf6-69c3-4f06-b90f-4437dbd0b4e4 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:44.632516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da37e03-8d0f-4be3-b4b3-871bc2cdde70 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:44.668009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f6458-9582-424f-9d98-049a5bb6e03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:44.668614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f6458-9582-424f-9d98-049a5bb6e03e None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:44.668889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f6458-9582-424f-9d98-049a5bb6e03e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:44.669155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f6458-9582-424f-9d98-049a5bb6e03e None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:44.669775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f6458-9582-424f-9d98-049a5bb6e03e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:44.670093 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f6458-9582-424f-9d98-049a5bb6e03e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:44.671621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f6458-9582-424f-9d98-049a5bb6e03e None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:44.676453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 387/773] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Sep 16 05:24:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:44.686343 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f6458-9582-424f-9d98-049a5bb6e03e None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:44.730222 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:44.781556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4baf6-69c3-4f06-b90f-4437dbd0b4e4 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:44.863205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfeba5-34a5-459c-b893-28156d7b71fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:44.863826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfeba5-34a5-459c-b893-28156d7b71fc None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:44.864098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfeba5-34a5-459c-b893-28156d7b71fc None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:44.864365 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfeba5-34a5-459c-b893-28156d7b71fc None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:44.864995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfeba5-34a5-459c-b893-28156d7b71fc None admin] RBAC: Adding query filter params (name=network) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:44.865292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfeba5-34a5-459c-b893-28156d7b71fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:44.870224 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfeba5-34a5-459c-b893-28156d7b71fc None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:44.879369 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 387/774] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 16 05:24:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:44.880569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d0da7-7a62-4090-89ba-b12fc1573832 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:44.880849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d0da7-7a62-4090-89ba-b12fc1573832 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:44.881117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d0da7-7a62-4090-89ba-b12fc1573832 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:44.882525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 388/775] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:44.884691 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f6458-9582-424f-9d98-049a5bb6e03e None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:44.888039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d0da7-7a62-4090-89ba-b12fc1573832 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:44.925403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:44.938321 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6182d-087a-4ac6-b4ad-35cebed14300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:44.938933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6182d-087a-4ac6-b4ad-35cebed14300 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:44.939206 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6182d-087a-4ac6-b4ad-35cebed14300 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:44.939535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6182d-087a-4ac6-b4ad-35cebed14300 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:44.940171 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6182d-087a-4ac6-b4ad-35cebed14300 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:44.940468 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6182d-087a-4ac6-b4ad-35cebed14300 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:44.941864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6182d-087a-4ac6-b4ad-35cebed14300 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:44.951758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 388/776] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Sep 16 05:24:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:44.953443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7673289-92ec-466d-ac1f-d97d30ffdc60 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:44.956511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7673289-92ec-466d-ac1f-d97d30ffdc60 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:44.956511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7673289-92ec-466d-ac1f-d97d30ffdc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:44.972793 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-f7673289-92ec-466d-ac1f-d97d30ffdc60 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:44.974146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f7673289-92ec-466d-ac1f-d97d30ffdc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:45.011143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d0da7-7a62-4090-89ba-b12fc1573832 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:45.051986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd969614-ff97-4c09-aeac-8452df0c2b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:45.052579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd969614-ff97-4c09-aeac-8452df0c2b53 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:45.052845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd969614-ff97-4c09-aeac-8452df0c2b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:45.053107 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd969614-ff97-4c09-aeac-8452df0c2b53 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:45.053721 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd969614-ff97-4c09-aeac-8452df0c2b53 None admin] RBAC: Adding query filter params (type=network) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:45.054123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd969614-ff97-4c09-aeac-8452df0c2b53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:45.055529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd969614-ff97-4c09-aeac-8452df0c2b53 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:45.056825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 389/777] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:24:45.057846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7673289-92ec-466d-ac1f-d97d30ffdc60 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:45.060820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 389/778] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 16 05:24:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:45.067591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd969614-ff97-4c09-aeac-8452df0c2b53 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:45.095730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:45.170022 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:45.187629 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd969614-ff97-4c09-aeac-8452df0c2b53 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:45.259085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e91ef4-94de-41e5-94e7-8a2b831c1632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:45.259705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e91ef4-94de-41e5-94e7-8a2b831c1632 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:45.259981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e91ef4-94de-41e5-94e7-8a2b831c1632 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:45.260251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e91ef4-94de-41e5-94e7-8a2b831c1632 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:45.260951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e91ef4-94de-41e5-94e7-8a2b831c1632 None admin] RBAC: Adding query filter params (interface=public, service_id=688b447c5f414fd4bd28dc95c3c4da54, region_id=RegionOne) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:45.261252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e91ef4-94de-41e5-94e7-8a2b831c1632 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:45.262634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e91ef4-94de-41e5-94e7-8a2b831c1632 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:45.267940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7673289-92ec-466d-ac1f-d97d30ffdc60 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:45.284766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 390/779] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 16 05:24:45 2022] GET /identity/v3/endpoints?service_id=688b447c5f414fd4bd28dc95c3c4da54&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:45.290040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e91ef4-94de-41e5-94e7-8a2b831c1632 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:45.340919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813525fa-207c-4603-a7b3-a4ffa4ffbc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:45.341578 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813525fa-207c-4603-a7b3-a4ffa4ffbc5d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:45.341920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813525fa-207c-4603-a7b3-a4ffa4ffbc5d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:45.342189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813525fa-207c-4603-a7b3-a4ffa4ffbc5d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:45.342991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813525fa-207c-4603-a7b3-a4ffa4ffbc5d None admin] RBAC: Adding query filter params (interface=public, service_id=a3e9328dfdf24a6e8d881ccb28fbab7a, region_id=RegionOne) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:45.343288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813525fa-207c-4603-a7b3-a4ffa4ffbc5d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:45.344643 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813525fa-207c-4603-a7b3-a4ffa4ffbc5d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:45.363798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 390/780] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 16 05:24:45 2022] GET /identity/v3/endpoints?service_id=a3e9328dfdf24a6e8d881ccb28fbab7a&interface=public®ion_id=RegionOne => generated 194 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:45.371064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813525fa-207c-4603-a7b3-a4ffa4ffbc5d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:45.408449 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e91ef4-94de-41e5-94e7-8a2b831c1632 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:45.451314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cde4dd-2615-49b5-858a-821305027f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:45.452344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cde4dd-2615-49b5-858a-821305027f43 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:45.452757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cde4dd-2615-49b5-858a-821305027f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:45.453026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cde4dd-2615-49b5-858a-821305027f43 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:45.453621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cde4dd-2615-49b5-858a-821305027f43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:45.455001 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cde4dd-2615-49b5-858a-821305027f43 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:45.460433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:45.461656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 391/781] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 16 05:24:45 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:45.470741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cde4dd-2615-49b5-858a-821305027f43 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:45.490088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813525fa-207c-4603-a7b3-a4ffa4ffbc5d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:45.527082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:45.542811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc9c28-4344-4706-9358-cc3ae38d9fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:45.542811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc9c28-4344-4706-9358-cc3ae38d9fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:45.547144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc9c28-4344-4706-9358-cc3ae38d9fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:45.547144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc9c28-4344-4706-9358-cc3ae38d9fa4 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:45.547144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc9c28-4344-4706-9358-cc3ae38d9fa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:45.547144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc9c28-4344-4706-9358-cc3ae38d9fa4 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:45.563710 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 391/782] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 16 05:24:45 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:45.648074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cde4dd-2615-49b5-858a-821305027f43 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:45.698623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60990f-0157-45e6-98b3-bac03bf9f66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:45.699271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60990f-0157-45e6-98b3-bac03bf9f66a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:45.699541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60990f-0157-45e6-98b3-bac03bf9f66a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:45.699810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60990f-0157-45e6-98b3-bac03bf9f66a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:45.714178 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60990f-0157-45e6-98b3-bac03bf9f66a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-ff60990f-0157-45e6-98b3-bac03bf9f66a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:45.719664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:45.720757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:45.721612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:45.721612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:45.721612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:45.721612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:45.721612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:45.721612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:45.721612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:45.721612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:45.721612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:45.721612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:45.721612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:45.721612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:45.721612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 392/783] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:24:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:45.726861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc9c28-4344-4706-9358-cc3ae38d9fa4 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:45.849947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc9c28-4344-4706-9358-cc3ae38d9fa4 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:45.902597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5872ce0-eeff-41c2-b68b-f8fc9b9f1d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:45.903211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5872ce0-eeff-41c2-b68b-f8fc9b9f1d32 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:45.903479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5872ce0-eeff-41c2-b68b-f8fc9b9f1d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:45.903740 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5872ce0-eeff-41c2-b68b-f8fc9b9f1d32 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:45.904411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5872ce0-eeff-41c2-b68b-f8fc9b9f1d32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:45.904702 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5872ce0-eeff-41c2-b68b-f8fc9b9f1d32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:45.906629 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5872ce0-eeff-41c2-b68b-f8fc9b9f1d32 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:45.919383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 392/784] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:45.931235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60990f-0157-45e6-98b3-bac03bf9f66a None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:46.043462 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:46.063340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60990f-0157-45e6-98b3-bac03bf9f66a None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:46.110842 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651023f7-c8a5-4b74-ab8d-31771ecd7ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:46.111849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651023f7-c8a5-4b74-ab8d-31771ecd7ccf None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:46.112346 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651023f7-c8a5-4b74-ab8d-31771ecd7ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:46.112818 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651023f7-c8a5-4b74-ab8d-31771ecd7ccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:46.128580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651023f7-c8a5-4b74-ab8d-31771ecd7ccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:46.130393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651023f7-c8a5-4b74-ab8d-31771ecd7ccf None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-651023f7-c8a5-4b74-ab8d-31771ecd7ccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:46.154859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:46.156458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:46.157360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:46.157360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:46.157360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:46.159692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 393/785] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:46.165290 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5872ce0-eeff-41c2-b68b-f8fc9b9f1d32 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:46.179983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:46.304639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac675b61-bdc3-45a2-9a01-da9fe622051e None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:46.305696 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac675b61-bdc3-45a2-9a01-da9fe622051e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:46.306208 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac675b61-bdc3-45a2-9a01-da9fe622051e None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:46.308277 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 394/786] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:46.314816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc65a6-2673-4bce-9de0-e731ec7469b2 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:46.315384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc65a6-2673-4bce-9de0-e731ec7469b2 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:46.315757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc65a6-2673-4bce-9de0-e731ec7469b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:46.319249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5872ce0-eeff-41c2-b68b-f8fc9b9f1d32 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:46.354681 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e0e60-ec40-44ef-a135-981510fb4e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:46.355525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e0e60-ec40-44ef-a135-981510fb4e53 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:46.355966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e0e60-ec40-44ef-a135-981510fb4e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:46.356081 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-76bc65a6-2673-4bce-9de0-e731ec7469b2 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:46.356470 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e0e60-ec40-44ef-a135-981510fb4e53 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:46.357244 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e0e60-ec40-44ef-a135-981510fb4e53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:46.357542 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-76bc65a6-2673-4bce-9de0-e731ec7469b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:46.357861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e0e60-ec40-44ef-a135-981510fb4e53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:46.360059 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e0e60-ec40-44ef-a135-981510fb4e53 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:46.374593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 393/787] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:46.379762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e0e60-ec40-44ef-a135-981510fb4e53 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:46.451685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 395/788] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:24:46.453228 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365838f-aca5-4b5a-9313-b08524370d45 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:46.453507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365838f-aca5-4b5a-9313-b08524370d45 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:46.453849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365838f-aca5-4b5a-9313-b08524370d45 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:46.455380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 396/789] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:46.462987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1efcc-ce6f-4710-a01e-e029fb9300ca None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:46.463412 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1efcc-ce6f-4710-a01e-e029fb9300ca None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:46.463835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd1efcc-ce6f-4710-a01e-e029fb9300ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:46.475239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:46.506057 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-6bd1efcc-ce6f-4710-a01e-e029fb9300ca None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:46.507484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6bd1efcc-ce6f-4710-a01e-e029fb9300ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:46.543214 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e0e60-ec40-44ef-a135-981510fb4e53 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:46.607334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e054540-5d37-4d01-908f-62c71ed3481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:46.608840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e054540-5d37-4d01-908f-62c71ed3481b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:46.609260 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e054540-5d37-4d01-908f-62c71ed3481b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:46.609665 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e054540-5d37-4d01-908f-62c71ed3481b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:46.627910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e054540-5d37-4d01-908f-62c71ed3481b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:46.629613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e054540-5d37-4d01-908f-62c71ed3481b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:46.643519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-5e054540-5d37-4d01-908f-62c71ed3481b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:46.652574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:46.653724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:46.655427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:46.655427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:46.655427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:46.655427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:46.655427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:46.655427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:46.655427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:46.655427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:46.655427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 05:24:46.655427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:46.661788 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 397/790] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:24:46.667622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1efcc-ce6f-4710-a01e-e029fb9300ca None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:46.668424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 394/791] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:24:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:46.768988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e054540-5d37-4d01-908f-62c71ed3481b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:46.796539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd1efcc-ce6f-4710-a01e-e029fb9300ca None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:46.885157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3327a-c374-4ce9-8a6e-627771c01498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:46.885767 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3327a-c374-4ce9-8a6e-627771c01498 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:46.886064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3327a-c374-4ce9-8a6e-627771c01498 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:46.886329 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada3327a-c374-4ce9-8a6e-627771c01498 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:46.887052 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3327a-c374-4ce9-8a6e-627771c01498 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:46.887349 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3327a-c374-4ce9-8a6e-627771c01498 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:46.888760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada3327a-c374-4ce9-8a6e-627771c01498 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:46.904083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e054540-5d37-4d01-908f-62c71ed3481b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:46.923535 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 398/792] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Sep 16 05:24:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:46.930200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3327a-c374-4ce9-8a6e-627771c01498 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:46.966196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:46.972808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21374f63-b773-4225-b979-bcda37a09064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:46.973444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21374f63-b773-4225-b979-bcda37a09064 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:46.973719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21374f63-b773-4225-b979-bcda37a09064 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:46.974015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21374f63-b773-4225-b979-bcda37a09064 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:47.002810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21374f63-b773-4225-b979-bcda37a09064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-21374f63-b773-4225-b979-bcda37a09064 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:47.012548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:47.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:47.014515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:47.014515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:47.014515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:47.014515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:47.014515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:47.014515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:47.014515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:47.014515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:47.014515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:47.014515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:47.014515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:47.014515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:47.015207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 395/793] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:47.017448 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21374f63-b773-4225-b979-bcda37a09064 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:47.031946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:47.094285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada3327a-c374-4ce9-8a6e-627771c01498 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:47.137464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72554250-5d45-4c32-8ca9-4317a7e7b119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:47.138042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72554250-5d45-4c32-8ca9-4317a7e7b119 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:47.138318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72554250-5d45-4c32-8ca9-4317a7e7b119 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:47.138588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72554250-5d45-4c32-8ca9-4317a7e7b119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:47.150626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72554250-5d45-4c32-8ca9-4317a7e7b119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:47.151450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21374f63-b773-4225-b979-bcda37a09064 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:47.152057 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72554250-5d45-4c32-8ca9-4317a7e7b119 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-72554250-5d45-4c32-8ca9-4317a7e7b119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:47.167835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:47.168826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:47.169714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:47.169714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:47.169714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:47.169714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 399/794] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:47.171595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72554250-5d45-4c32-8ca9-4317a7e7b119 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:47.194011 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb25a4-9ea8-4c67-811b-e6056ab99b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:47.194647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb25a4-9ea8-4c67-811b-e6056ab99b96 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:47.194927 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb25a4-9ea8-4c67-811b-e6056ab99b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:47.195641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb25a4-9ea8-4c67-811b-e6056ab99b96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:47.208768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb25a4-9ea8-4c67-811b-e6056ab99b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-bedb25a4-9ea8-4c67-811b-e6056ab99b96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:47.215598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:47.216754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:47.218384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:47.218384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:47.218384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:47.218384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:47.218384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:47.218384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:47.218384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:47.218384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:47.218384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:47.218384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:47.218384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:47.218384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:47.218384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 396/795] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:24:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:47.219251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb25a4-9ea8-4c67-811b-e6056ab99b96 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:47.277498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:47.290148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72554250-5d45-4c32-8ca9-4317a7e7b119 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:47.298672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:47.328883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff3201-7e78-4250-aac2-71da92864811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:47.329488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff3201-7e78-4250-aac2-71da92864811 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:47.329761 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff3201-7e78-4250-aac2-71da92864811 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:47.330053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff3201-7e78-4250-aac2-71da92864811 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:47.330707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff3201-7e78-4250-aac2-71da92864811 None admin] RBAC: Adding query filter params (name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:47.331006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff3201-7e78-4250-aac2-71da92864811 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:47.332371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff3201-7e78-4250-aac2-71da92864811 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:47.340946 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb25a4-9ea8-4c67-811b-e6056ab99b96 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:47.342287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 400/796] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:47.343900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff3201-7e78-4250-aac2-71da92864811 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:47.387276 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf434c-5192-4bec-860f-9962b2242f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:47.387928 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf434c-5192-4bec-860f-9962b2242f72 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:47.388151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf434c-5192-4bec-860f-9962b2242f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:47.388423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf434c-5192-4bec-860f-9962b2242f72 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:47.389060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf434c-5192-4bec-860f-9962b2242f72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:47.389397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf434c-5192-4bec-860f-9962b2242f72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:47.391161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf434c-5192-4bec-860f-9962b2242f72 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:47.403594 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 397/797] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:47.404468 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf434c-5192-4bec-860f-9962b2242f72 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:47.458062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff3201-7e78-4250-aac2-71da92864811 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:47.495088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc30c5-4e81-47f9-811f-184a334d5f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:47.495989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc30c5-4e81-47f9-811f-184a334d5f78 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:47.496516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc30c5-4e81-47f9-811f-184a334d5f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:47.496972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc30c5-4e81-47f9-811f-184a334d5f78 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:47.497920 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc30c5-4e81-47f9-811f-184a334d5f78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:47.498461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc30c5-4e81-47f9-811f-184a334d5f78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:47.500107 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc30c5-4e81-47f9-811f-184a334d5f78 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:47.506360 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf434c-5192-4bec-860f-9962b2242f72 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:47.514906 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:47.515172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 401/798] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:47.517178 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc30c5-4e81-47f9-811f-184a334d5f78 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:47.542921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f07fd5-5143-4f2d-93c6-3e9eea2baacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:47.543549 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f07fd5-5143-4f2d-93c6-3e9eea2baacc None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:47.543826 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f07fd5-5143-4f2d-93c6-3e9eea2baacc None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:47.544094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f07fd5-5143-4f2d-93c6-3e9eea2baacc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:47.550766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:47.555446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f07fd5-5143-4f2d-93c6-3e9eea2baacc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:47.557430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f07fd5-5143-4f2d-93c6-3e9eea2baacc None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-e6f07fd5-5143-4f2d-93c6-3e9eea2baacc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:47.573016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:47.574049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:47.574991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:47.574991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:47.574991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:47.574991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 398/799] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:47.575318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f07fd5-5143-4f2d-93c6-3e9eea2baacc None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:47.633774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc30c5-4e81-47f9-811f-184a334d5f78 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:47.673649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389ed47-cd52-44dc-a455-5e7924e5737f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:47.675060 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389ed47-cd52-44dc-a455-5e7924e5737f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:47.675478 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389ed47-cd52-44dc-a455-5e7924e5737f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:47.675909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389ed47-cd52-44dc-a455-5e7924e5737f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:47.683354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f07fd5-5143-4f2d-93c6-3e9eea2baacc None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:47.690086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389ed47-cd52-44dc-a455-5e7924e5737f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:47.691787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389ed47-cd52-44dc-a455-5e7924e5737f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-7389ed47-cd52-44dc-a455-5e7924e5737f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:47.705332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:47.706594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:47.707637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:47.707637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:47.707637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:47.707637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:47.707637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:47.707637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:47.707637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:47.707637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:47.709769 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 402/800] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:47.710890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389ed47-cd52-44dc-a455-5e7924e5737f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:47.726456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3815c5-af75-488d-a988-56758db52023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:47.727224 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3815c5-af75-488d-a988-56758db52023 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:47.727619 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3815c5-af75-488d-a988-56758db52023 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:47.728087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3815c5-af75-488d-a988-56758db52023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:47.740216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3815c5-af75-488d-a988-56758db52023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:47.741890 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3815c5-af75-488d-a988-56758db52023 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-3b3815c5-af75-488d-a988-56758db52023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:47.757573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:47.758792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:47.759877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:47.759877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:47.759877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:47.761092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 399/801] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:47.766960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3815c5-af75-488d-a988-56758db52023 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:47.781472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:47.817132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:47.827195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389ed47-cd52-44dc-a455-5e7924e5737f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:47.866420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cffa36-72ce-41d2-98f5-811f781ae3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:47.867390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cffa36-72ce-41d2-98f5-811f781ae3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:47.867810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cffa36-72ce-41d2-98f5-811f781ae3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:47.868225 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cffa36-72ce-41d2-98f5-811f781ae3c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:47.868991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cffa36-72ce-41d2-98f5-811f781ae3c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:47.869433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cffa36-72ce-41d2-98f5-811f781ae3c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:47.871159 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cffa36-72ce-41d2-98f5-811f781ae3c3 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:47.873547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3815c5-af75-488d-a988-56758db52023 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:47.885231 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 403/802] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:47.886315 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cffa36-72ce-41d2-98f5-811f781ae3c3 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:47.910832 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe427a-866c-434a-a1dc-09232872351f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:47.911530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe427a-866c-434a-a1dc-09232872351f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:47.911893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe427a-866c-434a-a1dc-09232872351f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:47.912482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe427a-866c-434a-a1dc-09232872351f None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:47.912975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe427a-866c-434a-a1dc-09232872351f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:47.913340 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe427a-866c-434a-a1dc-09232872351f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:47.915000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe427a-866c-434a-a1dc-09232872351f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:47.928119 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 400/803] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:47.928875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe427a-866c-434a-a1dc-09232872351f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:48.002388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:48.009277 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cffa36-72ce-41d2-98f5-811f781ae3c3 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:48.041499 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe427a-866c-434a-a1dc-09232872351f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:48.046081 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36322c54-0648-4bda-95ab-07a77455b14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:48.047019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36322c54-0648-4bda-95ab-07a77455b14c None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:48.047474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36322c54-0648-4bda-95ab-07a77455b14c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:48.047952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36322c54-0648-4bda-95ab-07a77455b14c None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:48.048799 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36322c54-0648-4bda-95ab-07a77455b14c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:48.049286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36322c54-0648-4bda-95ab-07a77455b14c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:48.051231 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36322c54-0648-4bda-95ab-07a77455b14c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:48.066884 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 404/804] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:48.068072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36322c54-0648-4bda-95ab-07a77455b14c None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:48.087611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:48.111843 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065970-f0e4-4ae1-9d06-1f718d8ea8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:48.112951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065970-f0e4-4ae1-9d06-1f718d8ea8cc None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:48.113798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065970-f0e4-4ae1-9d06-1f718d8ea8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:48.114510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065970-f0e4-4ae1-9d06-1f718d8ea8cc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:48.123257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92065970-f0e4-4ae1-9d06-1f718d8ea8cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:48.125388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92065970-f0e4-4ae1-9d06-1f718d8ea8cc None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-92065970-f0e4-4ae1-9d06-1f718d8ea8cc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:48.137726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:48.138939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:48.141285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:48.141285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:48.141285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:48.141285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:48.141285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:48.141285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:48.141285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:48.141285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:48.141285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 05:24:48.141285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:48.142905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 401/805] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 16 05:24:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:48.143876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065970-f0e4-4ae1-9d06-1f718d8ea8cc None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:48.201206 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36322c54-0648-4bda-95ab-07a77455b14c None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:48.236398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31788a-4ded-44dc-a486-399375cee9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:48.237051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31788a-4ded-44dc-a486-399375cee9bb None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:48.237324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31788a-4ded-44dc-a486-399375cee9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:48.237590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31788a-4ded-44dc-a486-399375cee9bb None admin] PATH_INFO: `/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/b260fed0d99a4d3fa8f9c2ad79223219/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:48.239505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31788a-4ded-44dc-a486-399375cee9bb None admin] RBAC: Authorizing `identity:create_grant(project_id=29b2e27932984de79ee8dfbe95312ac2, user_id=b260fed0d99a4d3fa8f9c2ad79223219, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:48.240916 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31788a-4ded-44dc-a486-399375cee9bb None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:48.245027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065970-f0e4-4ae1-9d06-1f718d8ea8cc None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:48.272628 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:48.278200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66373d92-6756-4a41-abf9-411acf03f400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:48.278815 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66373d92-6756-4a41-abf9-411acf03f400 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:48.279088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66373d92-6756-4a41-abf9-411acf03f400 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:48.279352 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66373d92-6756-4a41-abf9-411acf03f400 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:48.286673 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66373d92-6756-4a41-abf9-411acf03f400 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:48.288127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66373d92-6756-4a41-abf9-411acf03f400 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-66373d92-6756-4a41-abf9-411acf03f400 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:48.299605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:48.305411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:48.306631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:48.306631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:48.306631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:48.306631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:48.306631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:48.306631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:48.306631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:48.306631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:48.306631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:24:48.306631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:48.306631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 402/806] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 16 05:24:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:48.306631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66373d92-6756-4a41-abf9-411acf03f400 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:48.323740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 405/807] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:24:48 2022] PUT /identity/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/b260fed0d99a4d3fa8f9c2ad79223219/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:24:48.324865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c384-6cac-436a-9105-46822cf32def None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:48.325151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c384-6cac-436a-9105-46822cf32def None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:48.325420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d749c384-6cac-436a-9105-46822cf32def None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:48.326693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 406/808] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:48.333904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe077c5-531d-43a3-91e7-8396ee3180f8 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:48.333904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe077c5-531d-43a3-91e7-8396ee3180f8 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:48.333904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe077c5-531d-43a3-91e7-8396ee3180f8 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:48.334345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 407/809] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:48.337776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe077c5-531d-43a3-91e7-8396ee3180f8 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:48.414576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66373d92-6756-4a41-abf9-411acf03f400 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:48.419514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:48.451042 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29acae-dd26-4540-83b7-784e0257d38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:48.451725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29acae-dd26-4540-83b7-784e0257d38a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:48.452482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29acae-dd26-4540-83b7-784e0257d38a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:48.452738 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29acae-dd26-4540-83b7-784e0257d38a None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:48.453478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29acae-dd26-4540-83b7-784e0257d38a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:48.453781 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29acae-dd26-4540-83b7-784e0257d38a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:48.455292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29acae-dd26-4540-83b7-784e0257d38a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:48.457166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe077c5-531d-43a3-91e7-8396ee3180f8 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:48.477373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 403/810] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Sep 16 05:24:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:48.479029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb542e-25d3-46b4-a7ee-3cd70934d805 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:48.479309 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb542e-25d3-46b4-a7ee-3cd70934d805 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:48.479574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb542e-25d3-46b4-a7ee-3cd70934d805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:48.503331 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c2eb542e-25d3-46b4-a7ee-3cd70934d805 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:48.504457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c2eb542e-25d3-46b4-a7ee-3cd70934d805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:48.507400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0ffb2-77de-41ae-af59-2e114ac77845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:48.508178 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0ffb2-77de-41ae-af59-2e114ac77845 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:48.508602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0ffb2-77de-41ae-af59-2e114ac77845 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:48.509027 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0ffb2-77de-41ae-af59-2e114ac77845 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:48.510241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0ffb2-77de-41ae-af59-2e114ac77845 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:48.510241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0ffb2-77de-41ae-af59-2e114ac77845 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:48.511555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0ffb2-77de-41ae-af59-2e114ac77845 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:48.534012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 408/811] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 16 05:24:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:48.535239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d857e5-5f59-4f46-8c49-26bc89c56fd7 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:48.535519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d857e5-5f59-4f46-8c49-26bc89c56fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:48.535782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d857e5-5f59-4f46-8c49-26bc89c56fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:48.553974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-13d857e5-5f59-4f46-8c49-26bc89c56fd7 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:48.555143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-13d857e5-5f59-4f46-8c49-26bc89c56fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:48.558705 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:48.582097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 404/812] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:24:48.585423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb542e-25d3-46b4-a7ee-3cd70934d805 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:48.630944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 409/813] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:24:48.632256 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d857e5-5f59-4f46-8c49-26bc89c56fd7 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:48.708767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb542e-25d3-46b4-a7ee-3cd70934d805 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:48.742042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:48.748200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb43877-5890-4419-8288-8374b6252b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:48.749219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb43877-5890-4419-8288-8374b6252b74 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:48.749784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb43877-5890-4419-8288-8374b6252b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:48.750224 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb43877-5890-4419-8288-8374b6252b74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:48.761479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb43877-5890-4419-8288-8374b6252b74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:48.763231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb43877-5890-4419-8288-8374b6252b74 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-1bb43877-5890-4419-8288-8374b6252b74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:48.781990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:48.783409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:48.784298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:48.784298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:48.784298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:48.786933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 405/814] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:48.792695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb43877-5890-4419-8288-8374b6252b74 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:48.838217 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:48.878483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d857e5-5f59-4f46-8c49-26bc89c56fd7 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:48.907447 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb43877-5890-4419-8288-8374b6252b74 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:48.917090 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6382c2-4cc6-4591-92d3-5fee3ade56e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:48.917702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6382c2-4cc6-4591-92d3-5fee3ade56e0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:48.918008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6382c2-4cc6-4591-92d3-5fee3ade56e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:48.918279 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6382c2-4cc6-4591-92d3-5fee3ade56e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:48.953743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32347932-6282-45bc-87b0-13a75e56a442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:48.955781 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32347932-6282-45bc-87b0-13a75e56a442 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:48.956681 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32347932-6282-45bc-87b0-13a75e56a442 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:48.957166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32347932-6282-45bc-87b0-13a75e56a442 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:48.958227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32347932-6282-45bc-87b0-13a75e56a442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:48.958669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32347932-6282-45bc-87b0-13a75e56a442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:48.960506 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32347932-6282-45bc-87b0-13a75e56a442 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:48.981699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 406/815] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:48.983604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32347932-6282-45bc-87b0-13a75e56a442 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:49.002205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6382c2-4cc6-4591-92d3-5fee3ade56e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:49.005077 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6382c2-4cc6-4591-92d3-5fee3ade56e0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:49.079593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:49.086337 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:49.096415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32347932-6282-45bc-87b0-13a75e56a442 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:49.128389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 410/816] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:24:49.129582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6382c2-4cc6-4591-92d3-5fee3ade56e0 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:49.145032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5aeaf-ba1c-4a6d-aec3-6b7bcc885de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:49.146064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5aeaf-ba1c-4a6d-aec3-6b7bcc885de9 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:49.146354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5aeaf-ba1c-4a6d-aec3-6b7bcc885de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:49.146746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5aeaf-ba1c-4a6d-aec3-6b7bcc885de9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:49.147473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5aeaf-ba1c-4a6d-aec3-6b7bcc885de9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:49.149265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5aeaf-ba1c-4a6d-aec3-6b7bcc885de9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-c2d5aeaf-ba1c-4a6d-aec3-6b7bcc885de9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:49.162739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:49.163765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 16 05:24:49.164613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:49.168758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 407/817] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:48 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:49.169592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5aeaf-ba1c-4a6d-aec3-6b7bcc885de9 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:49.234930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6382c2-4cc6-4591-92d3-5fee3ade56e0 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:49.274273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ec39a-c8bc-42ed-90fe-1a5a12bfaab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:49.274907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ec39a-c8bc-42ed-90fe-1a5a12bfaab1 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:49.275182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ec39a-c8bc-42ed-90fe-1a5a12bfaab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:49.275413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5aeaf-ba1c-4a6d-aec3-6b7bcc885de9 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:49.275534 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ec39a-c8bc-42ed-90fe-1a5a12bfaab1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:49.282614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ec39a-c8bc-42ed-90fe-1a5a12bfaab1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:49.284230 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ec39a-c8bc-42ed-90fe-1a5a12bfaab1 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-853ec39a-c8bc-42ed-90fe-1a5a12bfaab1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:49.296708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:49.297623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:49.298554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:49.298554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:49.298554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:49.298554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:49.298554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:49.298554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:49.298554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:49.298554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:49.298554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 411/818] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:49.298554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ec39a-c8bc-42ed-90fe-1a5a12bfaab1 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:49.317406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a791b20-ef93-403c-aa08-99cc2e594814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:49.318215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a791b20-ef93-403c-aa08-99cc2e594814 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:49.318494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a791b20-ef93-403c-aa08-99cc2e594814 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:49.318945 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:49.319070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a791b20-ef93-403c-aa08-99cc2e594814 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:49.319620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a791b20-ef93-403c-aa08-99cc2e594814 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:49.320986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a791b20-ef93-403c-aa08-99cc2e594814 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-5a791b20-ef93-403c-aa08-99cc2e594814 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:49.331387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:49.332622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:49.333493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 408/819] 10.222.0.49 () {56 vars in 1129 bytes} [Fri Sep 16 05:24:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:49.337894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a791b20-ef93-403c-aa08-99cc2e594814 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:49.354931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:49.407991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ec39a-c8bc-42ed-90fe-1a5a12bfaab1 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:49.446650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a791b20-ef93-403c-aa08-99cc2e594814 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:49.446881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afc129-ac26-49f5-9db0-2ee2bd6ea983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:49.447336 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afc129-ac26-49f5-9db0-2ee2bd6ea983 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:49.447608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afc129-ac26-49f5-9db0-2ee2bd6ea983 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:49.447879 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afc129-ac26-49f5-9db0-2ee2bd6ea983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:49.459552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afc129-ac26-49f5-9db0-2ee2bd6ea983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:49.461518 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afc129-ac26-49f5-9db0-2ee2bd6ea983 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-67afc129-ac26-49f5-9db0-2ee2bd6ea983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:49.478403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:49.479402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:49.480302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:49.480302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:49.480302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:49.480302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 412/820] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:49.481065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afc129-ac26-49f5-9db0-2ee2bd6ea983 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:49.486736 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7e173-091f-4660-abbc-df4fe370a379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:49.487387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7e173-091f-4660-abbc-df4fe370a379 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:49.487662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7e173-091f-4660-abbc-df4fe370a379 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:49.487928 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7e173-091f-4660-abbc-df4fe370a379 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:49.488561 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7e173-091f-4660-abbc-df4fe370a379 None admin] RBAC: Adding query filter params (name=network) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:49.488858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7e173-091f-4660-abbc-df4fe370a379 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:49.490293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7e173-091f-4660-abbc-df4fe370a379 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:49.495138 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 409/821] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 16 05:24:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:49.495913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7e173-091f-4660-abbc-df4fe370a379 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:49.559017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:49.580478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:49.595290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afc129-ac26-49f5-9db0-2ee2bd6ea983 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:49.599375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7e173-091f-4660-abbc-df4fe370a379 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:49.634232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda0403-3459-4891-a244-8967fcc87b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:49.635212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda0403-3459-4891-a244-8967fcc87b62 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:49.635693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda0403-3459-4891-a244-8967fcc87b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:49.636166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88e051-0c64-4aaf-88ad-9f2645fe7f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:49.636393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda0403-3459-4891-a244-8967fcc87b62 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:49.636766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88e051-0c64-4aaf-88ad-9f2645fe7f1c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:49.637041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88e051-0c64-4aaf-88ad-9f2645fe7f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:49.637309 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88e051-0c64-4aaf-88ad-9f2645fe7f1c None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:49.637959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88e051-0c64-4aaf-88ad-9f2645fe7f1c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:49.638158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda0403-3459-4891-a244-8967fcc87b62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:49.638256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88e051-0c64-4aaf-88ad-9f2645fe7f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:49.638751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda0403-3459-4891-a244-8967fcc87b62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:49.639804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88e051-0c64-4aaf-88ad-9f2645fe7f1c None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:49.640853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda0403-3459-4891-a244-8967fcc87b62 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:49.644550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 410/822] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Sep 16 05:24:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:49.645338 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88e051-0c64-4aaf-88ad-9f2645fe7f1c None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:49.656714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 413/823] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:49.657922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda0403-3459-4891-a244-8967fcc87b62 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:49.755038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88e051-0c64-4aaf-88ad-9f2645fe7f1c None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:49.761981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda0403-3459-4891-a244-8967fcc87b62 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:49.792537 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:49.793032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038c8a5-c185-4653-9669-48559a2890ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:49.793635 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038c8a5-c185-4653-9669-48559a2890ac None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:49.793935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038c8a5-c185-4653-9669-48559a2890ac None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:49.794203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038c8a5-c185-4653-9669-48559a2890ac None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:49.794926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038c8a5-c185-4653-9669-48559a2890ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:49.795227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038c8a5-c185-4653-9669-48559a2890ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:49.796854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038c8a5-c185-4653-9669-48559a2890ac None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:49.799256 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec11e3-65c9-4c8b-885f-dc279089f48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:49.799845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec11e3-65c9-4c8b-885f-dc279089f48f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:49.800119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec11e3-65c9-4c8b-885f-dc279089f48f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:49.800383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec11e3-65c9-4c8b-885f-dc279089f48f None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:49.800994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec11e3-65c9-4c8b-885f-dc279089f48f None admin] RBAC: Adding query filter params (type=network) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:49.801283 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec11e3-65c9-4c8b-885f-dc279089f48f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:49.802636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec11e3-65c9-4c8b-885f-dc279089f48f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:49.808115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 414/824] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 16 05:24:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:49.808892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec11e3-65c9-4c8b-885f-dc279089f48f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:49.809910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 411/825] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:49.810846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038c8a5-c185-4653-9669-48559a2890ac None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:49.822849 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:49.918153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038c8a5-c185-4653-9669-48559a2890ac None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:49.930063 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec11e3-65c9-4c8b-885f-dc279089f48f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:49.955448 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028303b-098c-46ee-a8bc-965b210a3d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:49.956376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028303b-098c-46ee-a8bc-965b210a3d21 None admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:49.956840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028303b-098c-46ee-a8bc-965b210a3d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:49.957114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028303b-098c-46ee-a8bc-965b210a3d21 None admin] PATH_INFO: `/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/ba7a3df6fb6a4eb3a077a21b40794754/roles/ee551384189f433bba7e06f47a794cf0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:49.967038 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17d8e6-8a56-4f90-8ec6-d4738ada92e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:49.967775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17d8e6-8a56-4f90-8ec6-d4738ada92e6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:49.968271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17d8e6-8a56-4f90-8ec6-d4738ada92e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:49.968591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17d8e6-8a56-4f90-8ec6-d4738ada92e6 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:49.969214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17d8e6-8a56-4f90-8ec6-d4738ada92e6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:49.969510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17d8e6-8a56-4f90-8ec6-d4738ada92e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:49.970890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c17d8e6-8a56-4f90-8ec6-d4738ada92e6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:49.971770 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028303b-098c-46ee-a8bc-965b210a3d21 None admin] RBAC: Authorizing `identity:create_grant(project_id=29b2e27932984de79ee8dfbe95312ac2, user_id=ba7a3df6fb6a4eb3a077a21b40794754, role_id=ee551384189f433bba7e06f47a794cf0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:49.973193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028303b-098c-46ee-a8bc-965b210a3d21 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:49.975949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 415/826] 10.222.0.49 () {56 vars in 1124 bytes} [Fri Sep 16 05:24:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:49.981055 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17d8e6-8a56-4f90-8ec6-d4738ada92e6 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:50.028992 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 412/827] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:24:49 2022] PUT /identity/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/ba7a3df6fb6a4eb3a077a21b40794754/roles/ee551384189f433bba7e06f47a794cf0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:24:50.029940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028303b-098c-46ee-a8bc-965b210a3d21 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:50.058051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:50.104575 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:50.124013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17d8e6-8a56-4f90-8ec6-d4738ada92e6 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:50.156937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028303b-098c-46ee-a8bc-965b210a3d21 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:50.176585 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d16b59-e3e9-439b-9ff7-dc04627cb3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:50.177494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d16b59-e3e9-439b-9ff7-dc04627cb3da None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:50.178166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d16b59-e3e9-439b-9ff7-dc04627cb3da None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:50.178446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d16b59-e3e9-439b-9ff7-dc04627cb3da None admin] PATH_INFO: `/v3/endpoints` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:50.179134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d16b59-e3e9-439b-9ff7-dc04627cb3da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:50.182623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d16b59-e3e9-439b-9ff7-dc04627cb3da None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:50.199194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ea6b8-6533-4a4d-8d45-17cfd709df5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:50.200115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ea6b8-6533-4a4d-8d45-17cfd709df5a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:50.200491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ea6b8-6533-4a4d-8d45-17cfd709df5a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:50.200804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ea6b8-6533-4a4d-8d45-17cfd709df5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:50.209468 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ea6b8-6533-4a4d-8d45-17cfd709df5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:50.211218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ea6b8-6533-4a4d-8d45-17cfd709df5a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-ae2ea6b8-6533-4a4d-8d45-17cfd709df5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:50.225158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:50.226342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:50.227593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:50.227593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:50.227593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:50.227593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:50.227593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:50.227593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:50.227593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:50.227593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:50.229426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 413/828] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:50.229426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ea6b8-6533-4a4d-8d45-17cfd709df5a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:50.296785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 416/829] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:24:49 2022] POST /identity/v3/endpoints => generated 337 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:24:50.298642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d16b59-e3e9-439b-9ff7-dc04627cb3da None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:50.371298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ea6b8-6533-4a4d-8d45-17cfd709df5a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:50.407755 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:50.441624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d16b59-e3e9-439b-9ff7-dc04627cb3da None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:50.457796 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:50.485212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408ee3d-f547-4b40-b2c4-8e28d36f8a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:50.486104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408ee3d-f547-4b40-b2c4-8e28d36f8a6e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:50.486221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0eecb8-b296-4eaf-958e-9064638bdcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:50.486632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408ee3d-f547-4b40-b2c4-8e28d36f8a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:50.487018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0eecb8-b296-4eaf-958e-9064638bdcff None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:50.487230 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408ee3d-f547-4b40-b2c4-8e28d36f8a6e None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:50.487498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0eecb8-b296-4eaf-958e-9064638bdcff None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:50.487852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0eecb8-b296-4eaf-958e-9064638bdcff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:50.488361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408ee3d-f547-4b40-b2c4-8e28d36f8a6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:50.488705 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0eecb8-b296-4eaf-958e-9064638bdcff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:50.488977 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408ee3d-f547-4b40-b2c4-8e28d36f8a6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:50.490284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0eecb8-b296-4eaf-958e-9064638bdcff None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:50.490710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408ee3d-f547-4b40-b2c4-8e28d36f8a6e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:50.505489 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 417/830] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:50.511839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408ee3d-f547-4b40-b2c4-8e28d36f8a6e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:50.607043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 414/831] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:24:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:24:50.609068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021773df-a33e-41f2-97fb-80dd2cd8c814 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:50.609367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021773df-a33e-41f2-97fb-80dd2cd8c814 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:50.609626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021773df-a33e-41f2-97fb-80dd2cd8c814 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:50.617191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 415/832] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:50.619767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae6a4c-8821-448b-ac20-064f2e5a1ea0 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:50.620052 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae6a4c-8821-448b-ac20-064f2e5a1ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:50.620307 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae6a4c-8821-448b-ac20-064f2e5a1ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:50.642502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-10ae6a4c-8821-448b-ac20-064f2e5a1ea0 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:50.643790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-10ae6a4c-8821-448b-ac20-064f2e5a1ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:50.654418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408ee3d-f547-4b40-b2c4-8e28d36f8a6e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:50.702779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000838a-fa45-4e91-b58b-278069a07ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:50.704034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000838a-fa45-4e91-b58b-278069a07ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:50.704521 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000838a-fa45-4e91-b58b-278069a07ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:50.705023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000838a-fa45-4e91-b58b-278069a07ce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:50.706038 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000838a-fa45-4e91-b58b-278069a07ce6 None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=7db75c20e9744feb8b5dc78c554f4977) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:50.706584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000838a-fa45-4e91-b58b-278069a07ce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:50.708401 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000838a-fa45-4e91-b58b-278069a07ce6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:50.715646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 418/833] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:50 2022] GET /identity/v3/role_assignments?user.id=7db75c20e9744feb8b5dc78c554f4977&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:50.742052 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 416/834] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:24:51.013932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000838a-fa45-4e91-b58b-278069a07ce6 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:51.048826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:51.121874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000838a-fa45-4e91-b58b-278069a07ce6 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:51.159244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785a5c1-3b23-4e30-81c9-8a5c155bd358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:51.159868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785a5c1-3b23-4e30-81c9-8a5c155bd358 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:51.160135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785a5c1-3b23-4e30-81c9-8a5c155bd358 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:51.160399 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785a5c1-3b23-4e30-81c9-8a5c155bd358 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:51.171707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785a5c1-3b23-4e30-81c9-8a5c155bd358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-3785a5c1-3b23-4e30-81c9-8a5c155bd358 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:51.180830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:51.181778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:51.182675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:51.182675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:51.182675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:51.182675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:51.182675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:51.182675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:51.182675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:51.182675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:51.182675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:51.182675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:51.182675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:24:51.182675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:51.182675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 419/835] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:24:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:51.187933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae6a4c-8821-448b-ac20-064f2e5a1ea0 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:51.290588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae6a4c-8821-448b-ac20-064f2e5a1ea0 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:51.292232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:51.325721 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbad652-4d19-496b-81e2-3f1f2f5d24d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:51.327184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbad652-4d19-496b-81e2-3f1f2f5d24d8 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:51.327627 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbad652-4d19-496b-81e2-3f1f2f5d24d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:51.327902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbad652-4d19-496b-81e2-3f1f2f5d24d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:51.328631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbad652-4d19-496b-81e2-3f1f2f5d24d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:51.329300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbad652-4d19-496b-81e2-3f1f2f5d24d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:51.330833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbad652-4d19-496b-81e2-3f1f2f5d24d8 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:51.339582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 417/836] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:24:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:51.345300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785a5c1-3b23-4e30-81c9-8a5c155bd358 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:51.445479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:51.455801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785a5c1-3b23-4e30-81c9-8a5c155bd358 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:51.491552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e687248-cb9e-44d2-8476-4fee631672f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:51.492177 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e687248-cb9e-44d2-8476-4fee631672f0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:51.492445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e687248-cb9e-44d2-8476-4fee631672f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:51.492707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e687248-cb9e-44d2-8476-4fee631672f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:51.503285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e687248-cb9e-44d2-8476-4fee631672f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:51.504715 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e687248-cb9e-44d2-8476-4fee631672f0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-8e687248-cb9e-44d2-8476-4fee631672f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:51.520310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:51.521307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:51.522214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:51.522214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:51.522214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:51.522214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 420/837] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:51.528570 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbad652-4d19-496b-81e2-3f1f2f5d24d8 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:51.639048 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbad652-4d19-496b-81e2-3f1f2f5d24d8 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:51.673542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffb6e2-e8a4-4c01-ae7f-3507aa0abdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:51.674161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb6e2-e8a4-4c01-ae7f-3507aa0abdf9 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:51.674430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb6e2-e8a4-4c01-ae7f-3507aa0abdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:51.674694 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb6e2-e8a4-4c01-ae7f-3507aa0abdf9 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:51.675318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffb6e2-e8a4-4c01-ae7f-3507aa0abdf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:51.675668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffb6e2-e8a4-4c01-ae7f-3507aa0abdf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:51.677012 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffb6e2-e8a4-4c01-ae7f-3507aa0abdf9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:51.690154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 418/838] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:51.695666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e687248-cb9e-44d2-8476-4fee631672f0 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:51.802539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e687248-cb9e-44d2-8476-4fee631672f0 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:51.836514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256164d-e3ba-4948-9e8a-127a59fa9439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:51.837139 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256164d-e3ba-4948-9e8a-127a59fa9439 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:51.837408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256164d-e3ba-4948-9e8a-127a59fa9439 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:51.837669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256164d-e3ba-4948-9e8a-127a59fa9439 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:51.845986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256164d-e3ba-4948-9e8a-127a59fa9439 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:51.847410 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3256164d-e3ba-4948-9e8a-127a59fa9439 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-3256164d-e3ba-4948-9e8a-127a59fa9439 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:51.858771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:51.859701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:51.860642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:51.860642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:51.860642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:51.860642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:51.860642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:51.860642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:51.860642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:51.860642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:51.860642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 16 05:24:51.860642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:51.860642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 421/839] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:24:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:51.867384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffb6e2-e8a4-4c01-ae7f-3507aa0abdf9 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:51.894669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:51.973173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffb6e2-e8a4-4c01-ae7f-3507aa0abdf9 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:52.009271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f2f2b-4bc9-4715-a30b-394a988be333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:52.010244 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f2f2b-4bc9-4715-a30b-394a988be333 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:52.010716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f2f2b-4bc9-4715-a30b-394a988be333 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:52.011179 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f2f2b-4bc9-4715-a30b-394a988be333 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:52.012089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f2f2b-4bc9-4715-a30b-394a988be333 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:52.012611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f2f2b-4bc9-4715-a30b-394a988be333 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:52.014847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f2f2b-4bc9-4715-a30b-394a988be333 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:52.035422 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 419/840] 10.222.0.49 () {56 vars in 1135 bytes} [Fri Sep 16 05:24:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:52.041310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256164d-e3ba-4948-9e8a-127a59fa9439 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:52.050145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:52.151060 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256164d-e3ba-4948-9e8a-127a59fa9439 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:52.185295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeaa47e-3836-4044-ba13-c2214c4ce1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:52.185922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaa47e-3836-4044-ba13-c2214c4ce1bf None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:52.186245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaa47e-3836-4044-ba13-c2214c4ce1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:52.186511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeaa47e-3836-4044-ba13-c2214c4ce1bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:52.262102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeaa47e-3836-4044-ba13-c2214c4ce1bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:52.263459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeaa47e-3836-4044-ba13-c2214c4ce1bf None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:52.278701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:52.372944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 422/841] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:24:52.379188 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f2f2b-4bc9-4715-a30b-394a988be333 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:52.409868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f40d8-1c5f-4138-af56-3e33647948b0 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:52.410205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f40d8-1c5f-4138-af56-3e33647948b0 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:52.410426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f40d8-1c5f-4138-af56-3e33647948b0 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:52.412236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 423/842] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:24:52.417191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb9d09-63f5-4c66-8830-a29d42fc3cd9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:52.417544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb9d09-63f5-4c66-8830-a29d42fc3cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:52.417721 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb9d09-63f5-4c66-8830-a29d42fc3cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:52.437574 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-7cfb9d09-63f5-4c66-8830-a29d42fc3cd9 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:52.438763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7cfb9d09-63f5-4c66-8830-a29d42fc3cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:52.518830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 424/843] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:24:52.602238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:52.621375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f2f2b-4bc9-4715-a30b-394a988be333 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:52.656608 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc070-2c53-4151-8409-947dfe8379b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:52.657679 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc070-2c53-4151-8409-947dfe8379b5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:52.658176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc070-2c53-4151-8409-947dfe8379b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:52.658631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc070-2c53-4151-8409-947dfe8379b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:52.668942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bc070-2c53-4151-8409-947dfe8379b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:52.671359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bc070-2c53-4151-8409-947dfe8379b5 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-cc3bc070-2c53-4151-8409-947dfe8379b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:52.686653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:52.687739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:52.688637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:52.688637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:52.688637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:52.690661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 420/844] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:52.696348 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb9d09-63f5-4c66-8830-a29d42fc3cd9 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:52.728327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:52.805323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb9d09-63f5-4c66-8830-a29d42fc3cd9 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:52.812399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc070-2c53-4151-8409-947dfe8379b5 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:52.856568 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d38f8-e7ef-4008-aebd-ae7d07c87805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:52.857227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d38f8-e7ef-4008-aebd-ae7d07c87805 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:52.857501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d38f8-e7ef-4008-aebd-ae7d07c87805 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:52.857764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d38f8-e7ef-4008-aebd-ae7d07c87805 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:52.859845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d38f8-e7ef-4008-aebd-ae7d07c87805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:52.860153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d38f8-e7ef-4008-aebd-ae7d07c87805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:52.861504 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d38f8-e7ef-4008-aebd-ae7d07c87805 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:52.876545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 425/845] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:52.881891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d38f8-e7ef-4008-aebd-ae7d07c87805 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:52.935097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc070-2c53-4151-8409-947dfe8379b5 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:52.974466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d74cb-3d53-42b5-84ed-74f0064217b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:52.975334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d74cb-3d53-42b5-84ed-74f0064217b3 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:52.975780 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d74cb-3d53-42b5-84ed-74f0064217b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:52.976206 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d74cb-3d53-42b5-84ed-74f0064217b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:52.978167 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:52.987970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:52.989814 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d74cb-3d53-42b5-84ed-74f0064217b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-414d74cb-3d53-42b5-84ed-74f0064217b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:52.994184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:52.995228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:52.996094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:52.996094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:52.996094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:52.996094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:52.996094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:52.996094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:52.996094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:52.996094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:52.996094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:52.996094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:52.996094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:24:52.996094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:52.996094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 421/846] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:24:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:24:52.997611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc06ee-86ff-4d4a-8e4d-42b332107af0 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:52.997933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc06ee-86ff-4d4a-8e4d-42b332107af0 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:52.998199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc06ee-86ff-4d4a-8e4d-42b332107af0 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:53.000343 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 422/847] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:53.003242 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d38f8-e7ef-4008-aebd-ae7d07c87805 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:53.004481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc06ee-86ff-4d4a-8e4d-42b332107af0 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:53.041757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651352a1-a823-441c-8c1b-514be40b8f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:53.043045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651352a1-a823-441c-8c1b-514be40b8f3e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:53.043316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651352a1-a823-441c-8c1b-514be40b8f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:53.043578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651352a1-a823-441c-8c1b-514be40b8f3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:53.051090 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651352a1-a823-441c-8c1b-514be40b8f3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:53.052504 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651352a1-a823-441c-8c1b-514be40b8f3e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-651352a1-a823-441c-8c1b-514be40b8f3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:53.064491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:53.065364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:53.066649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:53.066649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:53.066649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:53.066649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:53.066649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:53.066649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:53.066649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:53.066649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:53.070252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 426/848] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:24:53.071328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e7145-cae7-45cd-a295-f2b7164d184a None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:53.071604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e7145-cae7-45cd-a295-f2b7164d184a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:53.071895 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e7145-cae7-45cd-a295-f2b7164d184a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:53.090454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f46e7145-cae7-45cd-a295-f2b7164d184a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:53.091708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f46e7145-cae7-45cd-a295-f2b7164d184a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:53.118069 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc06ee-86ff-4d4a-8e4d-42b332107af0 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:53.152148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f20d0d4-3913-40eb-a57a-65fd14c3b08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:53.152740 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20d0d4-3913-40eb-a57a-65fd14c3b08c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:53.153010 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20d0d4-3913-40eb-a57a-65fd14c3b08c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:53.153265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f20d0d4-3913-40eb-a57a-65fd14c3b08c None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:53.153949 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20d0d4-3913-40eb-a57a-65fd14c3b08c None admin] RBAC: Adding query filter params (name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:53.154253 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20d0d4-3913-40eb-a57a-65fd14c3b08c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:53.156738 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f20d0d4-3913-40eb-a57a-65fd14c3b08c None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:53.161712 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 427/849] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:24:53.162544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e7145-cae7-45cd-a295-f2b7164d184a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:53.164678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 423/850] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:24:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:53.165940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60326d2c-0313-404f-a779-4f266efea01e None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:53.166292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60326d2c-0313-404f-a779-4f266efea01e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:53.166630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60326d2c-0313-404f-a779-4f266efea01e None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:53.168014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 424/851] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:53.173639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60326d2c-0313-404f-a779-4f266efea01e None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:53.237988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:53.276125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:53.285646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60326d2c-0313-404f-a779-4f266efea01e None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:53.296183 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e7145-cae7-45cd-a295-f2b7164d184a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:53.320418 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e076db-52f2-4636-92d2-3d48f09e5b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:53.320966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e076db-52f2-4636-92d2-3d48f09e5b1d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:53.321235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e076db-52f2-4636-92d2-3d48f09e5b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:53.321496 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e076db-52f2-4636-92d2-3d48f09e5b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:53.333025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cab93e-c000-45ff-9be7-825fedc23502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:53.333514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e076db-52f2-4636-92d2-3d48f09e5b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:53.333623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cab93e-c000-45ff-9be7-825fedc23502 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:53.333917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cab93e-c000-45ff-9be7-825fedc23502 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:53.334185 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cab93e-c000-45ff-9be7-825fedc23502 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:53.334876 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cab93e-c000-45ff-9be7-825fedc23502 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:53.335063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e076db-52f2-4636-92d2-3d48f09e5b1d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:53.335161 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cab93e-c000-45ff-9be7-825fedc23502 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:53.336540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cab93e-c000-45ff-9be7-825fedc23502 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:53.348253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 428/852] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-e2e076db-52f2-4636-92d2-3d48f09e5b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:53.349272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:53.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:53.351135 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:53.351135 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:53.351135 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:53.351306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f3f3d-2f67-4705-957b-f8a209f09347 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:53.351306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f3f3d-2f67-4705-957b-f8a209f09347 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:53.351306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f3f3d-2f67-4705-957b-f8a209f09347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:53.352621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 425/853] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:53.355251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e076db-52f2-4636-92d2-3d48f09e5b1d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:53.374809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-903f3f3d-2f67-4705-957b-f8a209f09347 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:53.376279 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-903f3f3d-2f67-4705-957b-f8a209f09347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:53.454309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 429/854] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:24:53.455134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f3f3d-2f67-4705-957b-f8a209f09347 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:53.460711 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e076db-52f2-4636-92d2-3d48f09e5b1d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:53.472691 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:53.496791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b364b-b42a-43e1-a1b1-a3014f6c2b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:53.498584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b364b-b42a-43e1-a1b1-a3014f6c2b80 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:53.498584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b364b-b42a-43e1-a1b1-a3014f6c2b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:53.498584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b364b-b42a-43e1-a1b1-a3014f6c2b80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:53.499391 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b364b-b42a-43e1-a1b1-a3014f6c2b80 None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=b260fed0d99a4d3fa8f9c2ad79223219) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:53.499391 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b364b-b42a-43e1-a1b1-a3014f6c2b80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:53.500530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b364b-b42a-43e1-a1b1-a3014f6c2b80 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:53.506357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 426/855] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:53 2022] GET /identity/v3/role_assignments?user.id=b260fed0d99a4d3fa8f9c2ad79223219&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:53.506966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b364b-b42a-43e1-a1b1-a3014f6c2b80 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:53.553645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:53.599209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f3f3d-2f67-4705-957b-f8a209f09347 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:53.627949 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b364b-b42a-43e1-a1b1-a3014f6c2b80 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:53.644737 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666bce8-18ef-4890-aacf-79e39cb63620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:53.645361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666bce8-18ef-4890-aacf-79e39cb63620 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:53.645635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666bce8-18ef-4890-aacf-79e39cb63620 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:53.645945 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5666bce8-18ef-4890-aacf-79e39cb63620 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:53.646605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666bce8-18ef-4890-aacf-79e39cb63620 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:53.646972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666bce8-18ef-4890-aacf-79e39cb63620 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:53.648345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5666bce8-18ef-4890-aacf-79e39cb63620 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:53.666249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 430/856] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:53.667295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666bce8-18ef-4890-aacf-79e39cb63620 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:53.672732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401af4fa-06d1-4894-be62-61460cb727b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:53.673726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401af4fa-06d1-4894-be62-61460cb727b4 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:53.674242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401af4fa-06d1-4894-be62-61460cb727b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:53.674712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401af4fa-06d1-4894-be62-61460cb727b4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:53.675502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401af4fa-06d1-4894-be62-61460cb727b4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:53.677074 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401af4fa-06d1-4894-be62-61460cb727b4 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-401af4fa-06d1-4894-be62-61460cb727b4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:53.687436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:53.688458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 05:24:53.689310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:53.691266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 427/857] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:24:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:53.697768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401af4fa-06d1-4894-be62-61460cb727b4 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:53.741049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:53.785586 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5666bce8-18ef-4890-aacf-79e39cb63620 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:53.817562 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:53.818166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401af4fa-06d1-4894-be62-61460cb727b4 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:53.825582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cca8bc-f077-4463-81e9-d5ce5b37310e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:53.830499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cca8bc-f077-4463-81e9-d5ce5b37310e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:53.830499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cca8bc-f077-4463-81e9-d5ce5b37310e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:53.830499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cca8bc-f077-4463-81e9-d5ce5b37310e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:53.835622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cca8bc-f077-4463-81e9-d5ce5b37310e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:53.837125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cca8bc-f077-4463-81e9-d5ce5b37310e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-87cca8bc-f077-4463-81e9-d5ce5b37310e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:53.848607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:53.849892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:53.850999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:53.850999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:53.850999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:53.850999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:53.850999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:53.850999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:53.850999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:53.850999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:53.850999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 05:24:53.850999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:53.853139 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 431/858] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 16 05:24:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:53.854733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cca8bc-f077-4463-81e9-d5ce5b37310e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:53.862948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e4048-90ff-4acb-873f-1e029104ab76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:53.863559 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e4048-90ff-4acb-873f-1e029104ab76 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:53.863833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e4048-90ff-4acb-873f-1e029104ab76 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:53.864212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e4048-90ff-4acb-873f-1e029104ab76 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:53.864848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e4048-90ff-4acb-873f-1e029104ab76 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:53.865148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e4048-90ff-4acb-873f-1e029104ab76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:53.866544 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e4048-90ff-4acb-873f-1e029104ab76 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:53.876909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 428/859] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:24:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:53.876909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e4048-90ff-4acb-873f-1e029104ab76 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:53.981601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cca8bc-f077-4463-81e9-d5ce5b37310e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:53.988185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:53.991484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e4048-90ff-4acb-873f-1e029104ab76 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:54.018557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e021c2-7cdb-4a8c-bca9-f387dfa24710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:54.019184 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e021c2-7cdb-4a8c-bca9-f387dfa24710 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:54.019460 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e021c2-7cdb-4a8c-bca9-f387dfa24710 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:54.019731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e021c2-7cdb-4a8c-bca9-f387dfa24710 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:54.020308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e021c2-7cdb-4a8c-bca9-f387dfa24710 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:54.021663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e021c2-7cdb-4a8c-bca9-f387dfa24710 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:54.029891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc94fd-84b5-4cc7-acef-a8f9726a0ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:54.030722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc94fd-84b5-4cc7-acef-a8f9726a0ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-b8e021c2-7cdb-4a8c-bca9-f387dfa24710 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:54.031101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:54.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:54.032963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 432/860] 10.222.0.49 () {56 vars in 1132 bytes} [Fri Sep 16 05:24:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:54.034257 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e021c2-7cdb-4a8c-bca9-f387dfa24710 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:54.034519 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc94fd-84b5-4cc7-acef-a8f9726a0ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:54.034937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc94fd-84b5-4cc7-acef-a8f9726a0ee2 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:54.035670 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc94fd-84b5-4cc7-acef-a8f9726a0ee2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:54.035969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc94fd-84b5-4cc7-acef-a8f9726a0ee2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:54.037510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc94fd-84b5-4cc7-acef-a8f9726a0ee2 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:54.056627 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 429/861] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Sep 16 05:24:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:54.057059 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc94fd-84b5-4cc7-acef-a8f9726a0ee2 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:54.098996 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:54.151607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e021c2-7cdb-4a8c-bca9-f387dfa24710 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:54.169583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc94fd-84b5-4cc7-acef-a8f9726a0ee2 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:54.192351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341edd3c-2fff-4bdf-88e0-b6cfe6c0d834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:54.192956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341edd3c-2fff-4bdf-88e0-b6cfe6c0d834 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:54.193229 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341edd3c-2fff-4bdf-88e0-b6cfe6c0d834 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:54.193494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341edd3c-2fff-4bdf-88e0-b6cfe6c0d834 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:54.194144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341edd3c-2fff-4bdf-88e0-b6cfe6c0d834 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:54.194441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341edd3c-2fff-4bdf-88e0-b6cfe6c0d834 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:54.195806 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341edd3c-2fff-4bdf-88e0-b6cfe6c0d834 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:54.200507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 433/862] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:24:54 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:54.206878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341edd3c-2fff-4bdf-88e0-b6cfe6c0d834 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:54.215982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b7dcf-1b83-4190-b47e-44ecf04a48f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:54.217101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b7dcf-1b83-4190-b47e-44ecf04a48f5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:54.217101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b7dcf-1b83-4190-b47e-44ecf04a48f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:54.217711 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b7dcf-1b83-4190-b47e-44ecf04a48f5 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:54.218640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b7dcf-1b83-4190-b47e-44ecf04a48f5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:54.219049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b7dcf-1b83-4190-b47e-44ecf04a48f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:54.220591 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b7dcf-1b83-4190-b47e-44ecf04a48f5 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:54.226490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 430/863] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 16 05:24:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:54.232093 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b7dcf-1b83-4190-b47e-44ecf04a48f5 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:54.262348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:54.329639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341edd3c-2fff-4bdf-88e0-b6cfe6c0d834 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:54.354073 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:54.376735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b82bd-7fa8-4d53-998b-7be38b14199f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:54.376999 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b7dcf-1b83-4190-b47e-44ecf04a48f5 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:54.377366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b82bd-7fa8-4d53-998b-7be38b14199f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:54.377641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b82bd-7fa8-4d53-998b-7be38b14199f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:54.377950 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b82bd-7fa8-4d53-998b-7be38b14199f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:54.424048 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0ca99-f8f9-414b-90ef-e57f381d02a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:54.424584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0ca99-f8f9-414b-90ef-e57f381d02a3 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:54.424860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0ca99-f8f9-414b-90ef-e57f381d02a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:54.425837 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0ca99-f8f9-414b-90ef-e57f381d02a3 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:54.425837 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0ca99-f8f9-414b-90ef-e57f381d02a3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:54.426130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0ca99-f8f9-414b-90ef-e57f381d02a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:54.427475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0ca99-f8f9-414b-90ef-e57f381d02a3 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:54.433158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 431/864] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 16 05:24:54 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:54.485556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b82bd-7fa8-4d53-998b-7be38b14199f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:54.488318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b82bd-7fa8-4d53-998b-7be38b14199f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:54.621990 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 434/865] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:54 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:24:54.628520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0ca99-f8f9-414b-90ef-e57f381d02a3 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:54.726642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:54.743831 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0ca99-f8f9-414b-90ef-e57f381d02a3 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:54.780471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3623fb2-73d8-417c-959f-430be0cab6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:54.781474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3623fb2-73d8-417c-959f-430be0cab6cd None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:54.781747 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3623fb2-73d8-417c-959f-430be0cab6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:54.782046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3623fb2-73d8-417c-959f-430be0cab6cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:54.792533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3623fb2-73d8-417c-959f-430be0cab6cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:54.793995 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3623fb2-73d8-417c-959f-430be0cab6cd None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-b3623fb2-73d8-417c-959f-430be0cab6cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:54.811662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:54.812650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:54.813707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:54.813707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:54.813707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:54.813707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 432/866] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:54.818714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b82bd-7fa8-4d53-998b-7be38b14199f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:54.841691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:54.923505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b82bd-7fa8-4d53-998b-7be38b14199f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:54.963889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452937c0-f4e0-4d1b-92db-6711982b5de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:54.964491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452937c0-f4e0-4d1b-92db-6711982b5de2 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:54.964765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452937c0-f4e0-4d1b-92db-6711982b5de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:54.965030 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452937c0-f4e0-4d1b-92db-6711982b5de2 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:54.965673 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452937c0-f4e0-4d1b-92db-6711982b5de2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:54.965999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452937c0-f4e0-4d1b-92db-6711982b5de2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:54.967397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452937c0-f4e0-4d1b-92db-6711982b5de2 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:54.979651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 435/867] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:24:54.986895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3623fb2-73d8-417c-959f-430be0cab6cd None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:55.095832 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3623fb2-73d8-417c-959f-430be0cab6cd None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:55.130869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8805f4f-7b05-49c7-be3d-142e31fd8f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:55.131489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8805f4f-7b05-49c7-be3d-142e31fd8f4b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:55.131762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8805f4f-7b05-49c7-be3d-142e31fd8f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:55.132029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8805f4f-7b05-49c7-be3d-142e31fd8f4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:55.138840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8805f4f-7b05-49c7-be3d-142e31fd8f4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:55.140804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8805f4f-7b05-49c7-be3d-142e31fd8f4b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:55.142819 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-c8805f4f-7b05-49c7-be3d-142e31fd8f4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:24:55.152897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:55.153791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:55.160355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:24:55.160355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:24:55.160355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:24:55.160355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:24:55.160355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:24:55.160355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:24:55.160355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:24:55.160355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:24:55.160355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 433/868] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:55.160867 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452937c0-f4e0-4d1b-92db-6711982b5de2 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:55.251531 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:55.272058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452937c0-f4e0-4d1b-92db-6711982b5de2 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:55.307655 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa57f2e-add2-4fb0-924b-080531b6962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:55.308255 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa57f2e-add2-4fb0-924b-080531b6962d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:55.308525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa57f2e-add2-4fb0-924b-080531b6962d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:55.308790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa57f2e-add2-4fb0-924b-080531b6962d None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:55.309470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa57f2e-add2-4fb0-924b-080531b6962d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:55.309773 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa57f2e-add2-4fb0-924b-080531b6962d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:55.311852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa57f2e-add2-4fb0-924b-080531b6962d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:55.323947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 436/869] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:24:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:55.331750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8805f4f-7b05-49c7-be3d-142e31fd8f4b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:55.434886 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8805f4f-7b05-49c7-be3d-142e31fd8f4b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:55.473040 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e5432-2ff1-45f7-a284-80dda8e3a420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:55.474012 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e5432-2ff1-45f7-a284-80dda8e3a420 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:55.474537 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e5432-2ff1-45f7-a284-80dda8e3a420 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:55.475026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e5432-2ff1-45f7-a284-80dda8e3a420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:55.475933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e5432-2ff1-45f7-a284-80dda8e3a420 None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=ba7a3df6fb6a4eb3a077a21b40794754) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:55.476430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e5432-2ff1-45f7-a284-80dda8e3a420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:55.478054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e5432-2ff1-45f7-a284-80dda8e3a420 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:55.484714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 434/870] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:24:55 2022] GET /identity/v3/role_assignments?user.id=ba7a3df6fb6a4eb3a077a21b40794754&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:55.845361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb3133-02ee-4e0d-8647-3fda4e81af47 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:55.845742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb3133-02ee-4e0d-8647-3fda4e81af47 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:55.845965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb3133-02ee-4e0d-8647-3fda4e81af47 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:55.847659 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 437/871] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:55.854440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08233bae-c410-4fdb-ad65-31bd7cb32b63 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:55.855028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08233bae-c410-4fdb-ad65-31bd7cb32b63 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:55.855107 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08233bae-c410-4fdb-ad65-31bd7cb32b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:55.876895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-08233bae-c410-4fdb-ad65-31bd7cb32b63 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:55.878439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-08233bae-c410-4fdb-ad65-31bd7cb32b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:55.947651 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:55.959652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 435/872] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:24:56.254645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb3133-02ee-4e0d-8647-3fda4e81af47 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:56.379417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb3133-02ee-4e0d-8647-3fda4e81af47 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:56.418868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae29bd3-dca5-4b3c-8401-ee3f5c08d7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:56.419496 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae29bd3-dca5-4b3c-8401-ee3f5c08d7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:56.419767 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae29bd3-dca5-4b3c-8401-ee3f5c08d7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:56.420093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae29bd3-dca5-4b3c-8401-ee3f5c08d7b3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:56.420812 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae29bd3-dca5-4b3c-8401-ee3f5c08d7b3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:56.422484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae29bd3-dca5-4b3c-8401-ee3f5c08d7b3 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-0ae29bd3-dca5-4b3c-8401-ee3f5c08d7b3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:56.431987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:56.433017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:56.433917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 438/873] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 16 05:24:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:56.443150 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08233bae-c410-4fdb-ad65-31bd7cb32b63 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:56.563560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08233bae-c410-4fdb-ad65-31bd7cb32b63 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:56.611903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc427d-1108-4fd5-91b0-98c80529443c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:56.612517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc427d-1108-4fd5-91b0-98c80529443c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:56.612787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc427d-1108-4fd5-91b0-98c80529443c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:56.613051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc427d-1108-4fd5-91b0-98c80529443c None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:56.613686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc427d-1108-4fd5-91b0-98c80529443c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:56.614005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc427d-1108-4fd5-91b0-98c80529443c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:56.615361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc427d-1108-4fd5-91b0-98c80529443c None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:56.620529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 436/874] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 16 05:24:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:56.627220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae29bd3-dca5-4b3c-8401-ee3f5c08d7b3 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:56.633009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:56.694727 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec444d1-f1ea-4e3e-9566-b2838bd94ed4 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:56.695065 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec444d1-f1ea-4e3e-9566-b2838bd94ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:56.695259 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec444d1-f1ea-4e3e-9566-b2838bd94ed4 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:56.698119 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 437/875] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:56.704518 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34b4f9-e82e-4b32-86ee-ce651a4c6c1c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:56.704827 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34b4f9-e82e-4b32-86ee-ce651a4c6c1c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:56.705054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c34b4f9-e82e-4b32-86ee-ce651a4c6c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:56.723573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-2c34b4f9-e82e-4b32-86ee-ce651a4c6c1c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:56.724760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2c34b4f9-e82e-4b32-86ee-ce651a4c6c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:56.754899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae29bd3-dca5-4b3c-8401-ee3f5c08d7b3 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:56.796919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:56.800986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcc8de-49ef-4d53-bca3-0a5d5ced7b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:56.801732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcc8de-49ef-4d53-bca3-0a5d5ced7b23 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:56.802100 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcc8de-49ef-4d53-bca3-0a5d5ced7b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:56.802456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcc8de-49ef-4d53-bca3-0a5d5ced7b23 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:56.803153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcc8de-49ef-4d53-bca3-0a5d5ced7b23 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:56.803627 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcc8de-49ef-4d53-bca3-0a5d5ced7b23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:56.805376 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcc8de-49ef-4d53-bca3-0a5d5ced7b23 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:56.817939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 439/876] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 16 05:24:56 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:56.821849 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 438/877] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:24:56.922025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef97b3-e708-4ba1-9e51-b559abeb7480 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:56.922434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef97b3-e708-4ba1-9e51-b559abeb7480 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:56.922655 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef97b3-e708-4ba1-9e51-b559abeb7480 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:56.924350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 440/878] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:56.933981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5a0da-d2cc-4302-bdce-59ed16caba6c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:56.934974 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5a0da-d2cc-4302-bdce-59ed16caba6c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:56.935457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5a0da-d2cc-4302-bdce-59ed16caba6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:56.957851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-08e5a0da-d2cc-4302-bdce-59ed16caba6c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:56.959452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-08e5a0da-d2cc-4302-bdce-59ed16caba6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:57.051409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 439/879] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:24:57.115810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef97b3-e708-4ba1-9e51-b559abeb7480 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:57.187858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:57.223308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef97b3-e708-4ba1-9e51-b559abeb7480 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:57.258797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e7f71-9565-4f8f-a58c-afc994310984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:57.259422 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e7f71-9565-4f8f-a58c-afc994310984 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:57.259679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e7f71-9565-4f8f-a58c-afc994310984 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:57.259939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e7f71-9565-4f8f-a58c-afc994310984 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:57.260606 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e7f71-9565-4f8f-a58c-afc994310984 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:57.261997 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e7f71-9565-4f8f-a58c-afc994310984 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:57.336487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 441/880] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:24:57 2022] POST /identity/v3/services => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:24:57.340505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5a0da-d2cc-4302-bdce-59ed16caba6c None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:57.483693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5a0da-d2cc-4302-bdce-59ed16caba6c None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:57.504645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:57.534038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62d7c1-76cd-4843-a617-b0b0e303fc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:57.535185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62d7c1-76cd-4843-a617-b0b0e303fc4a None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:57.535682 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62d7c1-76cd-4843-a617-b0b0e303fc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:57.536221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62d7c1-76cd-4843-a617-b0b0e303fc4a None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:57.537425 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62d7c1-76cd-4843-a617-b0b0e303fc4a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:57.539845 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62d7c1-76cd-4843-a617-b0b0e303fc4a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:57.622480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 440/881] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:24:57 2022] POST /identity/v3/services => generated 269 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:24:57.980379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b07c7-cbf4-4a5d-b965-d65868bb9530 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:57.980703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b07c7-cbf4-4a5d-b965-d65868bb9530 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:57.980938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857b07c7-cbf4-4a5d-b965-d65868bb9530 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:57.982557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 442/882] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:57.989865 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b73a5-3091-42d3-bb75-b909d905fcbc None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:57.989865 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b73a5-3091-42d3-bb75-b909d905fcbc None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:57.989865 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b73a5-3091-42d3-bb75-b909d905fcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:58.008253 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-713b73a5-3091-42d3-bb75-b909d905fcbc None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:58.009706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-713b73a5-3091-42d3-bb75-b909d905fcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:58.088050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 441/883] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:24:58.356276 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b07c7-cbf4-4a5d-b965-d65868bb9530 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:58.471232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857b07c7-cbf4-4a5d-b965-d65868bb9530 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:58.498215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:58.506213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff667e6-d009-4443-9bde-683d1238cf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:58.510990 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff667e6-d009-4443-9bde-683d1238cf29 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:58.510990 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff667e6-d009-4443-9bde-683d1238cf29 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:58.510990 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff667e6-d009-4443-9bde-683d1238cf29 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:58.518978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff667e6-d009-4443-9bde-683d1238cf29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-dff667e6-d009-4443-9bde-683d1238cf29 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:24:58.523933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:58.524960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:58.526157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:58.526157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:58.526157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:58.526157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:58.526157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:24:58.526157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:24:58.526157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:24:58.526157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:24:58.526157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:24:58.526157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:24:58.526157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 05:24:58.526157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:58.526157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 443/884] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Sep 16 05:24:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:58.533591 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b73a5-3091-42d3-bb75-b909d905fcbc None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:58.705265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:58.709485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b73a5-3091-42d3-bb75-b909d905fcbc None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:58.752458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64240ab6-cb68-4cd6-a149-e3920fee4a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:58.753480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64240ab6-cb68-4cd6-a149-e3920fee4a98 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:58.753993 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64240ab6-cb68-4cd6-a149-e3920fee4a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:58.754469 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64240ab6-cb68-4cd6-a149-e3920fee4a98 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:58.755342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64240ab6-cb68-4cd6-a149-e3920fee4a98 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:58.755841 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64240ab6-cb68-4cd6-a149-e3920fee4a98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:58.762949 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64240ab6-cb68-4cd6-a149-e3920fee4a98 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:58.776248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 442/885] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 16 05:24:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:58.782906 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff667e6-d009-4443-9bde-683d1238cf29 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:58.895530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff667e6-d009-4443-9bde-683d1238cf29 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:58.954589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f669d19-556c-48c2-8473-a209a6fe49a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:58.955229 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f669d19-556c-48c2-8473-a209a6fe49a8 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:58.955553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f669d19-556c-48c2-8473-a209a6fe49a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:58.955763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f669d19-556c-48c2-8473-a209a6fe49a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:58.974388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f669d19-556c-48c2-8473-a209a6fe49a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:58.975841 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f669d19-556c-48c2-8473-a209a6fe49a8 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-6f669d19-556c-48c2-8473-a209a6fe49a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:24:58.996474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:58.997495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:24:58.998713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:24:58.998713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:24:58.998713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:59.001844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 444/886] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:59.006943 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64240ab6-cb68-4cd6-a149-e3920fee4a98 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:59.213039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64240ab6-cb68-4cd6-a149-e3920fee4a98 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:59.250339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088dcc2-869d-40a5-8b60-a9f19f284696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:59.250937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088dcc2-869d-40a5-8b60-a9f19f284696 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:59.251203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088dcc2-869d-40a5-8b60-a9f19f284696 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:59.251461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088dcc2-869d-40a5-8b60-a9f19f284696 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:59.252092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088dcc2-869d-40a5-8b60-a9f19f284696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:59.252383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088dcc2-869d-40a5-8b60-a9f19f284696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:59.253722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088dcc2-869d-40a5-8b60-a9f19f284696 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:59.270110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 443/887] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:59.274894 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f669d19-556c-48c2-8473-a209a6fe49a8 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:59.321150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:59.324085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d33ef3-1cfd-4b57-9ff3-5c8732099c0b None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:59.324367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d33ef3-1cfd-4b57-9ff3-5c8732099c0b None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:59.324636 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d33ef3-1cfd-4b57-9ff3-5c8732099c0b None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:59.326229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 444/888] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:59.332030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f360faf-85ac-40b5-b6c4-96812e8eefbd None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:59.337351 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f360faf-85ac-40b5-b6c4-96812e8eefbd None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:59.337351 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f360faf-85ac-40b5-b6c4-96812e8eefbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:59.357208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-0f360faf-85ac-40b5-b6c4-96812e8eefbd None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:24:59.359027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0f360faf-85ac-40b5-b6c4-96812e8eefbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:24:59.379284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:59.392165 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f669d19-556c-48c2-8473-a209a6fe49a8 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:59.436579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04972a8c-1c99-4bb5-8ce7-1ecbb209e982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:59.437614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04972a8c-1c99-4bb5-8ce7-1ecbb209e982 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:59.437946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04972a8c-1c99-4bb5-8ce7-1ecbb209e982 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:59.438217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04972a8c-1c99-4bb5-8ce7-1ecbb209e982 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:59.448925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04972a8c-1c99-4bb5-8ce7-1ecbb209e982 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:59.450648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04972a8c-1c99-4bb5-8ce7-1ecbb209e982 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:59.453886 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 445/889] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-04972a8c-1c99-4bb5-8ce7-1ecbb209e982 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:24:59.470024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:24:59.471193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:24:59.472164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:24:59.472164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:24:59.472164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:24:59.472164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:24:59.472164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:24:59.472164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:24:59.472164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:24:59.472164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:24:59.472164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 05:24:59.472164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:24:59.474330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 445/890] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 16 05:24:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:24:59.481863 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f360faf-85ac-40b5-b6c4-96812e8eefbd None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:59.599909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f360faf-85ac-40b5-b6c4-96812e8eefbd None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:59.636904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a9f4e-1b73-43eb-9bf3-d9d195d9e79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:59.637534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a9f4e-1b73-43eb-9bf3-d9d195d9e79f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:59.637807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a9f4e-1b73-43eb-9bf3-d9d195d9e79f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:59.638103 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a9f4e-1b73-43eb-9bf3-d9d195d9e79f None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:59.638842 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a9f4e-1b73-43eb-9bf3-d9d195d9e79f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:24:59.639677 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a9f4e-1b73-43eb-9bf3-d9d195d9e79f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:59.641411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a9f4e-1b73-43eb-9bf3-d9d195d9e79f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:59.663333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 446/891] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Sep 16 05:24:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:24:59.669212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04972a8c-1c99-4bb5-8ce7-1ecbb209e982 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:59.741277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a9f4e-1b73-43eb-9bf3-d9d195d9e79f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:24:59.758179 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:59.777141 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:24:59.781034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04972a8c-1c99-4bb5-8ce7-1ecbb209e982 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:59.825345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4859ac4-d263-4397-bbb7-7b205a49fff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:59.827176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4859ac4-d263-4397-bbb7-7b205a49fff0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:59.827508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4859ac4-d263-4397-bbb7-7b205a49fff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:59.827851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4859ac4-d263-4397-bbb7-7b205a49fff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:59.853999 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a9f4e-1b73-43eb-9bf3-d9d195d9e79f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:24:59.899760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348e2f3-5611-4014-9da5-dc267e5def7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:24:59.900641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348e2f3-5611-4014-9da5-dc267e5def7d None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:59.901085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348e2f3-5611-4014-9da5-dc267e5def7d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:59.901356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348e2f3-5611-4014-9da5-dc267e5def7d None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:59.902046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348e2f3-5611-4014-9da5-dc267e5def7d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:59.903405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a348e2f3-5611-4014-9da5-dc267e5def7d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:59.903795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4859ac4-d263-4397-bbb7-7b205a49fff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:24:59.905429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4859ac4-d263-4397-bbb7-7b205a49fff0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:24:59.966793 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 447/892] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:24:59 2022] POST /identity/v3/services => generated 241 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:24:59.968527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08572a37-b031-4cf7-bd53-a7c4595d8071 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:59.968804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08572a37-b031-4cf7-bd53-a7c4595d8071 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:59.969068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08572a37-b031-4cf7-bd53-a7c4595d8071 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:59.970344 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 448/893] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:59.971429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ed48b-8cf5-48ab-91d2-7d936b470d61 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:59.971688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ed48b-8cf5-48ab-91d2-7d936b470d61 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:59.971958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ed48b-8cf5-48ab-91d2-7d936b470d61 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:24:59.973409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 449/894] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:24:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:24:59.978799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cc74b-f6f9-4f00-9a82-d6f35678f3bf None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:24:59.979126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cc74b-f6f9-4f00-9a82-d6f35678f3bf None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:24:59.979340 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cc74b-f6f9-4f00-9a82-d6f35678f3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:00.005324 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-249cc74b-f6f9-4f00-9a82-d6f35678f3bf None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:00.005324 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-249cc74b-f6f9-4f00-9a82-d6f35678f3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:00.018683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:00.032522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 446/895] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:24:59 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:25:00.032522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c741db-ad62-4b60-a1d6-8c394541b87d None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:00.032522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c741db-ad62-4b60-a1d6-8c394541b87d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:00.032522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c741db-ad62-4b60-a1d6-8c394541b87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:00.088120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-47c741db-ad62-4b60-a1d6-8c394541b87d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:00.089351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-47c741db-ad62-4b60-a1d6-8c394541b87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:00.112983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:00.128421 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 450/896] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:24:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:00.129295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cc74b-f6f9-4f00-9a82-d6f35678f3bf None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:00.188495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 447/897] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:00.264828 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cc74b-f6f9-4f00-9a82-d6f35678f3bf None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:00.301897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7505402-4b12-4923-b66a-2ce315000705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:00.302930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7505402-4b12-4923-b66a-2ce315000705 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:00.303426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7505402-4b12-4923-b66a-2ce315000705 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:00.303888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7505402-4b12-4923-b66a-2ce315000705 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:00.314818 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7505402-4b12-4923-b66a-2ce315000705 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:00.316542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7505402-4b12-4923-b66a-2ce315000705 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-d7505402-4b12-4923-b66a-2ce315000705 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:25:00.332426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:00.333389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:25:00.334301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:25:00.334301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:00.334301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:00.336361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 451/898] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:00.342860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c741db-ad62-4b60-a1d6-8c394541b87d None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:00.432763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7505402-4b12-4923-b66a-2ce315000705 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:00.489164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:00.502044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c741db-ad62-4b60-a1d6-8c394541b87d None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:00.549367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:00.555331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65357a12-e990-4687-856f-d90882762fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:00.555937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65357a12-e990-4687-856f-d90882762fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:00.556206 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65357a12-e990-4687-856f-d90882762fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:00.556396 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7505402-4b12-4923-b66a-2ce315000705 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:00.556508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65357a12-e990-4687-856f-d90882762fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:00.557084 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65357a12-e990-4687-856f-d90882762fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:00.557375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65357a12-e990-4687-856f-d90882762fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:00.558797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65357a12-e990-4687-856f-d90882762fe0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:00.571222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 448/899] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:00.571848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65357a12-e990-4687-856f-d90882762fe0 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:00.604880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f82a55-354b-41ba-8345-5c896a6b0f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:00.605639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f82a55-354b-41ba-8345-5c896a6b0f7a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:00.606009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f82a55-354b-41ba-8345-5c896a6b0f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:00.606348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f82a55-354b-41ba-8345-5c896a6b0f7a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:00.606998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f82a55-354b-41ba-8345-5c896a6b0f7a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:00.608535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f82a55-354b-41ba-8345-5c896a6b0f7a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-30f82a55-354b-41ba-8345-5c896a6b0f7a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:00.618089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:00.622043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:00.629354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 452/900] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:25:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:00.630217 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f82a55-354b-41ba-8345-5c896a6b0f7a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:00.690210 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65357a12-e990-4687-856f-d90882762fe0 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:00.727947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6120d-4d23-45bf-baa0-872707d29560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:00.728498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6120d-4d23-45bf-baa0-872707d29560 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:00.728766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6120d-4d23-45bf-baa0-872707d29560 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:00.729027 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6120d-4d23-45bf-baa0-872707d29560 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:00.729592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6120d-4d23-45bf-baa0-872707d29560 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:00.730975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6120d-4d23-45bf-baa0-872707d29560 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:00.732848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f82a55-354b-41ba-8345-5c896a6b0f7a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-84f6120d-4d23-45bf-baa0-872707d29560 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:00.739941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:00.740914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:00.741899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 449/901] 10.222.0.49 () {56 vars in 1132 bytes} [Fri Sep 16 05:25:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:00.742785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6120d-4d23-45bf-baa0-872707d29560 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:00.750624 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:00.773495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d34942-90d4-407a-8975-336afbb18c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:00.774237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d34942-90d4-407a-8975-336afbb18c29 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:00.774572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d34942-90d4-407a-8975-336afbb18c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:00.774840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d34942-90d4-407a-8975-336afbb18c29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:00.782552 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d34942-90d4-407a-8975-336afbb18c29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:00.783963 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d34942-90d4-407a-8975-336afbb18c29 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-b8d34942-90d4-407a-8975-336afbb18c29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:25:00.795739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:00.796776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:00.798115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:25:00.798115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:25:00.798115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:25:00.798115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:25:00.798115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:25:00.798115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:25:00.798115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:00.798115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:00.801840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 453/902] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:25:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:00.802409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d34942-90d4-407a-8975-336afbb18c29 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:00.808361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:00.856763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6120d-4d23-45bf-baa0-872707d29560 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:00.894264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65011a-f27f-461c-9cd8-63b99c4e5f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:00.894900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65011a-f27f-461c-9cd8-63b99c4e5f65 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:00.895155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65011a-f27f-461c-9cd8-63b99c4e5f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:00.895415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65011a-f27f-461c-9cd8-63b99c4e5f65 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:00.896023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65011a-f27f-461c-9cd8-63b99c4e5f65 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:00.896318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65011a-f27f-461c-9cd8-63b99c4e5f65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:00.897657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65011a-f27f-461c-9cd8-63b99c4e5f65 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:00.902589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 450/903] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:25:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:00.908443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65011a-f27f-461c-9cd8-63b99c4e5f65 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:00.913128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d34942-90d4-407a-8975-336afbb18c29 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:00.949102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5460a-c295-4274-b5c3-2cd08a486ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:00.949726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5460a-c295-4274-b5c3-2cd08a486ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:00.950027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5460a-c295-4274-b5c3-2cd08a486ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:00.950295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5460a-c295-4274-b5c3-2cd08a486ae6 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:00.950910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5460a-c295-4274-b5c3-2cd08a486ae6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:00.951206 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5460a-c295-4274-b5c3-2cd08a486ae6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:00.952808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5460a-c295-4274-b5c3-2cd08a486ae6 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:00.957920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 454/904] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 16 05:25:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:00.962982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5460a-c295-4274-b5c3-2cd08a486ae6 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:00.999524 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:01.020474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65011a-f27f-461c-9cd8-63b99c4e5f65 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:01.038461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:01.056908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a22e-aaba-4912-8512-6217180940cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:01.057440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a22e-aaba-4912-8512-6217180940cb None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:01.057709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a22e-aaba-4912-8512-6217180940cb None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:01.057999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a22e-aaba-4912-8512-6217180940cb None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:01.058786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a22e-aaba-4912-8512-6217180940cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:01.059111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a22e-aaba-4912-8512-6217180940cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:01.060997 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a22e-aaba-4912-8512-6217180940cb None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:01.067617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5460a-c295-4274-b5c3-2cd08a486ae6 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:01.078819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 451/905] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:25:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:01.078819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a22e-aaba-4912-8512-6217180940cb None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:01.106909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4e9b9-fd19-4649-81eb-7e2a71713381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:01.107501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4e9b9-fd19-4649-81eb-7e2a71713381 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:01.108292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4e9b9-fd19-4649-81eb-7e2a71713381 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:01.108553 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd4e9b9-fd19-4649-81eb-7e2a71713381 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:01.109167 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4e9b9-fd19-4649-81eb-7e2a71713381 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:01.109463 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4e9b9-fd19-4649-81eb-7e2a71713381 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:01.110826 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd4e9b9-fd19-4649-81eb-7e2a71713381 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:01.115453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 455/906] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:25:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:01.115453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4e9b9-fd19-4649-81eb-7e2a71713381 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:01.190520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a22e-aaba-4912-8512-6217180940cb None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:01.226003 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc2d8a-39a8-48bb-940e-e60fd9abeebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:01.226642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:01.226754 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc2d8a-39a8-48bb-940e-e60fd9abeebc None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:01.227032 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc2d8a-39a8-48bb-940e-e60fd9abeebc None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:01.227297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc2d8a-39a8-48bb-940e-e60fd9abeebc None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:01.228388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc2d8a-39a8-48bb-940e-e60fd9abeebc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:01.228827 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc2d8a-39a8-48bb-940e-e60fd9abeebc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:01.230222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc2d8a-39a8-48bb-940e-e60fd9abeebc None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:01.230321 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd4e9b9-fd19-4649-81eb-7e2a71713381 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:01.241006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 452/907] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 16 05:25:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:01.241006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc2d8a-39a8-48bb-940e-e60fd9abeebc None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:01.272231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ea20c-9b1c-4c3d-a31c-0022b3d21ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:01.272832 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ea20c-9b1c-4c3d-a31c-0022b3d21ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:01.273101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ea20c-9b1c-4c3d-a31c-0022b3d21ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:01.273366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ea20c-9b1c-4c3d-a31c-0022b3d21ee8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:01.274104 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ea20c-9b1c-4c3d-a31c-0022b3d21ee8 None admin] RBAC: Adding query filter params (role.id=6851de0f376f41a8a000506d1df67181, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=ba7a3df6fb6a4eb3a077a21b40794754) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:01.274404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ea20c-9b1c-4c3d-a31c-0022b3d21ee8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:01.275808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ea20c-9b1c-4c3d-a31c-0022b3d21ee8 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:01.282451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 456/908] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:25:01 2022] GET /identity/v3/role_assignments?user.id=ba7a3df6fb6a4eb3a077a21b40794754&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=6851de0f376f41a8a000506d1df67181 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:01.283473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ea20c-9b1c-4c3d-a31c-0022b3d21ee8 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:01.299076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:01.373593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc2d8a-39a8-48bb-940e-e60fd9abeebc None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:01.419179 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b2395-1e54-4e5c-af61-bdf4f1351fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:01.420014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2395-1e54-4e5c-af61-bdf4f1351fdc None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:01.420337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2395-1e54-4e5c-af61-bdf4f1351fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:01.420606 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2395-1e54-4e5c-af61-bdf4f1351fdc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:01.421305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b2395-1e54-4e5c-af61-bdf4f1351fdc None admin] RBAC: Adding query filter params (interface=public, service_id=1ae4d79a2f164ec28aeaca893162f123, region_id=RegionOne) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:01.421605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b2395-1e54-4e5c-af61-bdf4f1351fdc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:01.423703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b2395-1e54-4e5c-af61-bdf4f1351fdc None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:01.430803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 453/909] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 16 05:25:01 2022] GET /identity/v3/endpoints?service_id=1ae4d79a2f164ec28aeaca893162f123&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:01.436917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b2395-1e54-4e5c-af61-bdf4f1351fdc None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:01.442597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ea20c-9b1c-4c3d-a31c-0022b3d21ee8 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:01.488294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad657fad-2581-40e7-90fe-07a065a8a8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:01.489733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad657fad-2581-40e7-90fe-07a065a8a8fe None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:01.490258 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad657fad-2581-40e7-90fe-07a065a8a8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:01.491202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad657fad-2581-40e7-90fe-07a065a8a8fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:01.492382 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad657fad-2581-40e7-90fe-07a065a8a8fe None admin] RBAC: Adding query filter params (interface=public, service_id=b4206476cee142818555d05fae5546e6, region_id=RegionOne) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:01.492890 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad657fad-2581-40e7-90fe-07a065a8a8fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:01.494770 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad657fad-2581-40e7-90fe-07a065a8a8fe None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:01.502718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 457/910] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 16 05:25:01 2022] GET /identity/v3/endpoints?service_id=b4206476cee142818555d05fae5546e6&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:01.508236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad657fad-2581-40e7-90fe-07a065a8a8fe None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:01.539037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:01.573941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b2395-1e54-4e5c-af61-bdf4f1351fdc None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:01.585967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:01.616127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad657fad-2581-40e7-90fe-07a065a8a8fe None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:01.622653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d39cca-1ff9-4ed5-bffa-fa91395dfabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:01.624530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d39cca-1ff9-4ed5-bffa-fa91395dfabf None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:01.624972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d39cca-1ff9-4ed5-bffa-fa91395dfabf None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:01.625399 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d39cca-1ff9-4ed5-bffa-fa91395dfabf None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:01.626538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d39cca-1ff9-4ed5-bffa-fa91395dfabf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:01.628523 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d39cca-1ff9-4ed5-bffa-fa91395dfabf None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:01.636657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 454/911] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 16 05:25:01 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:25:01.654066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321a802-85bc-42fb-b0d0-5d5eb17fe25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:01.656034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321a802-85bc-42fb-b0d0-5d5eb17fe25d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:01.656490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321a802-85bc-42fb-b0d0-5d5eb17fe25d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:01.656910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321a802-85bc-42fb-b0d0-5d5eb17fe25d None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:01.657706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321a802-85bc-42fb-b0d0-5d5eb17fe25d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:01.659406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321a802-85bc-42fb-b0d0-5d5eb17fe25d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:01.666785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 458/912] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 16 05:25:01 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:25:02.301738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25319858-259e-4d40-93a7-b2bdc77d3f41 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:02.302409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25319858-259e-4d40-93a7-b2bdc77d3f41 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:02.302915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25319858-259e-4d40-93a7-b2bdc77d3f41 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:02.304763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 455/913] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:02.309611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda35079-a09f-4ada-93c3-b0e929aebdf3 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:02.310078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda35079-a09f-4ada-93c3-b0e929aebdf3 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:02.310154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda35079-a09f-4ada-93c3-b0e929aebdf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:02.329404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-dda35079-a09f-4ada-93c3-b0e929aebdf3 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:02.330589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-dda35079-a09f-4ada-93c3-b0e929aebdf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:02.402022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 459/914] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:02.680808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25319858-259e-4d40-93a7-b2bdc77d3f41 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:02.790545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25319858-259e-4d40-93a7-b2bdc77d3f41 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:02.828884 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0f850-d869-491a-9310-169a660c2fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:02.829742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0f850-d869-491a-9310-169a660c2fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:02.830187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0f850-d869-491a-9310-169a660c2fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:02.830619 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0f850-d869-491a-9310-169a660c2fc5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:02.831335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0f850-d869-491a-9310-169a660c2fc5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:02.832840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0f850-d869-491a-9310-169a660c2fc5 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-75d0f850-d869-491a-9310-169a660c2fc5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:02.843441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:02.844680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:25:02.845741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:02.848050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 456/915] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 16 05:25:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:02.856009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda35079-a09f-4ada-93c3-b0e929aebdf3 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:02.893455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:02.965145 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda35079-a09f-4ada-93c3-b0e929aebdf3 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:03.003439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca0e96-4849-43ed-8fd1-a140cda8af06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:03.004434 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca0e96-4849-43ed-8fd1-a140cda8af06 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:03.004923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca0e96-4849-43ed-8fd1-a140cda8af06 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:03.005393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca0e96-4849-43ed-8fd1-a140cda8af06 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:03.006278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca0e96-4849-43ed-8fd1-a140cda8af06 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:03.006784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca0e96-4849-43ed-8fd1-a140cda8af06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:03.008383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca0e96-4849-43ed-8fd1-a140cda8af06 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:03.015089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 460/916] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 16 05:25:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:03.023242 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0f850-d869-491a-9310-169a660c2fc5 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:03.056483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:03.129236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0f850-d869-491a-9310-169a660c2fc5 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:03.166458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1be6f-0713-40ea-9f7c-c4e3b50e1ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:03.167419 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1be6f-0713-40ea-9f7c-c4e3b50e1ccc None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:03.167894 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1be6f-0713-40ea-9f7c-c4e3b50e1ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:03.168358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af1be6f-0713-40ea-9f7c-c4e3b50e1ccc None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:03.169185 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af1be6f-0713-40ea-9f7c-c4e3b50e1ccc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:03.169714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af1be6f-0713-40ea-9f7c-c4e3b50e1ccc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:03.171934 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af1be6f-0713-40ea-9f7c-c4e3b50e1ccc None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:03.177332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 457/917] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 16 05:25:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:03.183000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca0e96-4849-43ed-8fd1-a140cda8af06 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:03.274275 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:03.289858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca0e96-4849-43ed-8fd1-a140cda8af06 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:03.327566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073267d3-3c7d-48be-8536-64b497d4d2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:03.328198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073267d3-3c7d-48be-8536-64b497d4d2cc None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:03.328450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073267d3-3c7d-48be-8536-64b497d4d2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:03.328721 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073267d3-3c7d-48be-8536-64b497d4d2cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:03.329413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073267d3-3c7d-48be-8536-64b497d4d2cc None admin] RBAC: Adding query filter params (interface=public, service_id=37f365e213a44e72a65c4cd11ba1543f, region_id=RegionOne) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:03.329720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073267d3-3c7d-48be-8536-64b497d4d2cc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:03.331112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073267d3-3c7d-48be-8536-64b497d4d2cc None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:03.336537 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 461/918] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 16 05:25:03 2022] GET /identity/v3/endpoints?service_id=37f365e213a44e72a65c4cd11ba1543f&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:03.343663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1be6f-0713-40ea-9f7c-c4e3b50e1ccc None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:03.447275 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:03.451346 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af1be6f-0713-40ea-9f7c-c4e3b50e1ccc None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:03.489724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d7fbe-cfad-4f89-9a66-4ce8495926af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:03.490351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d7fbe-cfad-4f89-9a66-4ce8495926af None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:03.490621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d7fbe-cfad-4f89-9a66-4ce8495926af None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:03.490884 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d7fbe-cfad-4f89-9a66-4ce8495926af None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:03.491480 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d7fbe-cfad-4f89-9a66-4ce8495926af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:03.492960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d7fbe-cfad-4f89-9a66-4ce8495926af None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:03.506323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 458/919] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 16 05:25:03 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:25:03.570095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963fa37-e966-4738-afa0-68a49de356da None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:03.570281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963fa37-e966-4738-afa0-68a49de356da None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:03.570548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963fa37-e966-4738-afa0-68a49de356da None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:03.572193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 462/920] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:03.578092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a855-79a3-4698-978e-298493a03c5c None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:03.578370 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a855-79a3-4698-978e-298493a03c5c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:03.578640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5a855-79a3-4698-978e-298493a03c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:03.599725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0cd5a855-79a3-4698-978e-298493a03c5c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:03.600938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0cd5a855-79a3-4698-978e-298493a03c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:03.681839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 459/921] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:03.921108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249196d-e6a8-4804-9d4f-078562518c42 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:03.921752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249196d-e6a8-4804-9d4f-078562518c42 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:03.922198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249196d-e6a8-4804-9d4f-078562518c42 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:03.923870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 463/922] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:03.929367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fe067-f14b-4e17-baf4-545044ef239e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:03.929907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fe067-f14b-4e17-baf4-545044ef239e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:03.930389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fe067-f14b-4e17-baf4-545044ef239e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:03.949975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-7b9fe067-f14b-4e17-baf4-545044ef239e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:03.951575 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7b9fe067-f14b-4e17-baf4-545044ef239e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:03.958056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249196d-e6a8-4804-9d4f-078562518c42 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:04.029291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 460/923] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:04.085045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249196d-e6a8-4804-9d4f-078562518c42 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:04.090954 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c2099-1c57-4fca-b710-4934de8e4f84 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:04.091240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c2099-1c57-4fca-b710-4934de8e4f84 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:04.091514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c2099-1c57-4fca-b710-4934de8e4f84 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:04.093686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 461/924] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:25:04.098987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd79242-2d9d-4420-ab4a-6c24f04818df None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:04.099284 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd79242-2d9d-4420-ab4a-6c24f04818df None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:04.099554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd79242-2d9d-4420-ab4a-6c24f04818df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:04.126661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf7941-9e2d-47ad-9a89-470d9057f4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:04.127338 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf7941-9e2d-47ad-9a89-470d9057f4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:04.127611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf7941-9e2d-47ad-9a89-470d9057f4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:04.127879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf7941-9e2d-47ad-9a89-470d9057f4c2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:04.129371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0dd79242-2d9d-4420-ab4a-6c24f04818df None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:04.130658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0dd79242-2d9d-4420-ab4a-6c24f04818df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:04.141194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf7941-9e2d-47ad-9a89-470d9057f4c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-ffaf7941-9e2d-47ad-9a89-470d9057f4c2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:25:04.148785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:04.149948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:04.151063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:04.151063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:04.151063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:04.151063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:04.151063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:25:04.151063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:25:04.151063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:25:04.151063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:25:04.151063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:25:04.151063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:25:04.151063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 05:25:04.151063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:04.152362 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:04.155141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 464/925] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Sep 16 05:25:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:04.161155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf7941-9e2d-47ad-9a89-470d9057f4c2 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:04.174230 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:04.213947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 462/926] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:04.268170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf7941-9e2d-47ad-9a89-470d9057f4c2 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:04.302531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfbcd1-8685-466c-b86c-ff8fa36aeff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:04.303434 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfbcd1-8685-466c-b86c-ff8fa36aeff9 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:04.303848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfbcd1-8685-466c-b86c-ff8fa36aeff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:04.304253 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfbcd1-8685-466c-b86c-ff8fa36aeff9 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:04.305072 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfbcd1-8685-466c-b86c-ff8fa36aeff9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:04.305508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfbcd1-8685-466c-b86c-ff8fa36aeff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:04.307026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cfbcd1-8685-466c-b86c-ff8fa36aeff9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:04.315860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd79242-2d9d-4420-ab4a-6c24f04818df None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:04.316475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 465/927] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 16 05:25:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:04.322148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfbcd1-8685-466c-b86c-ff8fa36aeff9 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:04.398036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:04.429519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd79242-2d9d-4420-ab4a-6c24f04818df None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:04.439238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfbcd1-8685-466c-b86c-ff8fa36aeff9 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:04.467926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a60608-6458-4062-9164-0163c32107ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:04.468708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a60608-6458-4062-9164-0163c32107ab None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:04.469267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a60608-6458-4062-9164-0163c32107ab None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:04.469394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a60608-6458-4062-9164-0163c32107ab None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:04.470116 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a60608-6458-4062-9164-0163c32107ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:04.471580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a60608-6458-4062-9164-0163c32107ab None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:04.477771 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a82311-470b-4388-829d-f59fde60612b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:04.478687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a82311-470b-4388-829d-f59fde60612b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:04.479132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a82311-470b-4388-829d-f59fde60612b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:04.479591 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a82311-470b-4388-829d-f59fde60612b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-a1a60608-6458-4062-9164-0163c32107ab None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:04.481787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:04.482807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 16 05:25:04.483720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:04.488366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 463/928] 10.222.0.49 () {56 vars in 1132 bytes} [Fri Sep 16 05:25:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:04.489415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a60608-6458-4062-9164-0163c32107ab None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:04.496822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a82311-470b-4388-829d-f59fde60612b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:04.499170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a82311-470b-4388-829d-f59fde60612b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:04.510241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-62a82311-470b-4388-829d-f59fde60612b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:25:04.515252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:04.516397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:25:04.517656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:25:04.517656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:04.517656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:04.517656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 466/929] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:04.517931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a82311-470b-4388-829d-f59fde60612b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:04.608577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a60608-6458-4062-9164-0163c32107ab None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:04.627305 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a82311-470b-4388-829d-f59fde60612b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:04.650824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:04.653698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed9265-5999-427e-a438-6dab4f0767c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:04.654995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed9265-5999-427e-a438-6dab4f0767c6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:04.655364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed9265-5999-427e-a438-6dab4f0767c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:04.655722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed9265-5999-427e-a438-6dab4f0767c6 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:04.656438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed9265-5999-427e-a438-6dab4f0767c6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:04.657750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed9265-5999-427e-a438-6dab4f0767c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:04.658170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ed9265-5999-427e-a438-6dab4f0767c6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:04.667162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409bb0f-6042-4f87-ac12-3b8d3b0bb2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:04.667906 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409bb0f-6042-4f87-ac12-3b8d3b0bb2bd None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:04.668178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409bb0f-6042-4f87-ac12-3b8d3b0bb2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:04.670748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f409bb0f-6042-4f87-ac12-3b8d3b0bb2bd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:04.670748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409bb0f-6042-4f87-ac12-3b8d3b0bb2bd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:04.671176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f409bb0f-6042-4f87-ac12-3b8d3b0bb2bd None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-f409bb0f-6042-4f87-ac12-3b8d3b0bb2bd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:04.680113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:04.682056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:04.682919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 467/930] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:25:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:04.683712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409bb0f-6042-4f87-ac12-3b8d3b0bb2bd None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:04.694544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 464/931] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 16 05:25:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:04.694544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed9265-5999-427e-a438-6dab4f0767c6 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:04.780719 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:04.791769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f409bb0f-6042-4f87-ac12-3b8d3b0bb2bd None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:04.800593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed9265-5999-427e-a438-6dab4f0767c6 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:04.827056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a21220-88f5-4333-ac77-bfd24b2f66db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:04.827842 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a21220-88f5-4333-ac77-bfd24b2f66db None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:04.828114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a21220-88f5-4333-ac77-bfd24b2f66db None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:04.828394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a21220-88f5-4333-ac77-bfd24b2f66db None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:04.829089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a21220-88f5-4333-ac77-bfd24b2f66db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:04.829535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a21220-88f5-4333-ac77-bfd24b2f66db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:04.830730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a21220-88f5-4333-ac77-bfd24b2f66db None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:04.838744 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c542d28-c39f-40b4-9e23-5f50d53eed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:04.839113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c542d28-c39f-40b4-9e23-5f50d53eed9e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:04.839387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c542d28-c39f-40b4-9e23-5f50d53eed9e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:04.839651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c542d28-c39f-40b4-9e23-5f50d53eed9e None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:04.840261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c542d28-c39f-40b4-9e23-5f50d53eed9e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:04.840554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c542d28-c39f-40b4-9e23-5f50d53eed9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:04.842618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c542d28-c39f-40b4-9e23-5f50d53eed9e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:04.853569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 465/932] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:25:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:04.853729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 468/933] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:04.853811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c542d28-c39f-40b4-9e23-5f50d53eed9e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:04.854327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a21220-88f5-4333-ac77-bfd24b2f66db None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:04.905481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:04.960954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a21220-88f5-4333-ac77-bfd24b2f66db None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:04.970248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c542d28-c39f-40b4-9e23-5f50d53eed9e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:04.993683 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c17bf-ab37-42df-99e2-d8ffa56cce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:04.994371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c17bf-ab37-42df-99e2-d8ffa56cce1f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:04.994646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c17bf-ab37-42df-99e2-d8ffa56cce1f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:04.994908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c17bf-ab37-42df-99e2-d8ffa56cce1f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:05.002570 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c17bf-ab37-42df-99e2-d8ffa56cce1f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:05.004037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c17bf-ab37-42df-99e2-d8ffa56cce1f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:05.008796 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf27b09-2aed-4d2b-878d-a295b707e59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:05.009403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf27b09-2aed-4d2b-878d-a295b707e59c None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:05.009676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf27b09-2aed-4d2b-878d-a295b707e59c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:05.009972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf27b09-2aed-4d2b-878d-a295b707e59c None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:05.010754 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf27b09-2aed-4d2b-878d-a295b707e59c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:05.011051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf27b09-2aed-4d2b-878d-a295b707e59c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:05.012379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf27b09-2aed-4d2b-878d-a295b707e59c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-ce8c17bf-ab37-42df-99e2-d8ffa56cce1f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:25:05.015891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:05.017066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:05.023203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:05.023203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:05.023203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:25:05.023203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:25:05.023203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:25:05.023203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:25:05.023203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:25:05.023203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:25:05.023203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 05:25:05.023203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:05.023203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 469/934] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 16 05:25:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:05.023203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c17bf-ab37-42df-99e2-d8ffa56cce1f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:05.024494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 466/935] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 16 05:25:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:05.025297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf27b09-2aed-4d2b-878d-a295b707e59c None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:05.037567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:05.124420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c17bf-ab37-42df-99e2-d8ffa56cce1f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:05.129412 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf27b09-2aed-4d2b-878d-a295b707e59c None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:05.132148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:05.160996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4495f4-1f68-40b1-9f48-117e8ba41c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:05.161637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4495f4-1f68-40b1-9f48-117e8ba41c58 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:05.161937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4495f4-1f68-40b1-9f48-117e8ba41c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:05.162214 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4495f4-1f68-40b1-9f48-117e8ba41c58 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:05.162821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4495f4-1f68-40b1-9f48-117e8ba41c58 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:05.163601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4495f4-1f68-40b1-9f48-117e8ba41c58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:05.165205 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4495f4-1f68-40b1-9f48-117e8ba41c58 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:05.165515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf46677-1a02-40b3-b87d-31efaa0c7a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:05.166143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf46677-1a02-40b3-b87d-31efaa0c7a4e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:05.166416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf46677-1a02-40b3-b87d-31efaa0c7a4e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:05.166701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf46677-1a02-40b3-b87d-31efaa0c7a4e None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:05.167561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf46677-1a02-40b3-b87d-31efaa0c7a4e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:05.167664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf46677-1a02-40b3-b87d-31efaa0c7a4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:05.169082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf46677-1a02-40b3-b87d-31efaa0c7a4e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:05.171093 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 470/936] 10.222.0.49 () {56 vars in 1114 bytes} [Fri Sep 16 05:25:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:05.171871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4495f4-1f68-40b1-9f48-117e8ba41c58 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:05.197117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 467/937] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Sep 16 05:25:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:05.197117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf46677-1a02-40b3-b87d-31efaa0c7a4e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:05.267971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:05.279009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4495f4-1f68-40b1-9f48-117e8ba41c58 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:05.303610 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf46677-1a02-40b3-b87d-31efaa0c7a4e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:05.316248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d19818-f046-4e18-b8ad-e1ed5f4154f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:05.316875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d19818-f046-4e18-b8ad-e1ed5f4154f7 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:05.317144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d19818-f046-4e18-b8ad-e1ed5f4154f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:05.317412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d19818-f046-4e18-b8ad-e1ed5f4154f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:05.318095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d19818-f046-4e18-b8ad-e1ed5f4154f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:05.319451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d19818-f046-4e18-b8ad-e1ed5f4154f7 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:05.338819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094b942-92a4-4400-80c9-cdcc0556c0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:05.339415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094b942-92a4-4400-80c9-cdcc0556c0f9 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:05.339682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094b942-92a4-4400-80c9-cdcc0556c0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:05.340019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094b942-92a4-4400-80c9-cdcc0556c0f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:05.340667 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094b942-92a4-4400-80c9-cdcc0556c0f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:05.342559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094b942-92a4-4400-80c9-cdcc0556c0f9 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:05.407875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 471/938] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:25:05 2022] POST /identity/v3/endpoints => generated 347 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:25:05.408704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d19818-f046-4e18-b8ad-e1ed5f4154f7 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:05.445851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 468/939] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:25:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:25:05.475405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:05.575532 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d19818-f046-4e18-b8ad-e1ed5f4154f7 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:05.622254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210ef5c-f828-4a58-90e3-12ae19d94699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:05.623227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210ef5c-f828-4a58-90e3-12ae19d94699 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:05.627218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210ef5c-f828-4a58-90e3-12ae19d94699 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:05.627218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210ef5c-f828-4a58-90e3-12ae19d94699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:05.636149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210ef5c-f828-4a58-90e3-12ae19d94699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:05.637613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e210ef5c-f828-4a58-90e3-12ae19d94699 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-e210ef5c-f828-4a58-90e3-12ae19d94699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:25:05.655466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:05.656535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:25:05.657782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:25:05.657782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:05.657782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:05.657782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 472/940] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:05.662953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094b942-92a4-4400-80c9-cdcc0556c0f9 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:05.783601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094b942-92a4-4400-80c9-cdcc0556c0f9 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:05.819606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e17f86-557d-40b2-8596-bb44e50f4115 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:05.820530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e17f86-557d-40b2-8596-bb44e50f4115 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:05.821036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e17f86-557d-40b2-8596-bb44e50f4115 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:05.821676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:05.824273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 473/941] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:05.830816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009d2cb-96a9-401c-a83f-a6b8e8ea0e61 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:05.831080 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009d2cb-96a9-401c-a83f-a6b8e8ea0e61 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:05.831290 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009d2cb-96a9-401c-a83f-a6b8e8ea0e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:05.832482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea89a69-a390-45c3-8760-65a494cbdbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:05.833364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea89a69-a390-45c3-8760-65a494cbdbee None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:05.833797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea89a69-a390-45c3-8760-65a494cbdbee None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:05.834230 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea89a69-a390-45c3-8760-65a494cbdbee None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:05.839026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea89a69-a390-45c3-8760-65a494cbdbee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:05.839026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea89a69-a390-45c3-8760-65a494cbdbee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:05.839026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea89a69-a390-45c3-8760-65a494cbdbee None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:05.851653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-e009d2cb-96a9-401c-a83f-a6b8e8ea0e61 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:05.852318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 469/942] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:05.852930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e009d2cb-96a9-401c-a83f-a6b8e8ea0e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:05.858689 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea89a69-a390-45c3-8760-65a494cbdbee None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:05.938214 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:05.943728 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 474/943] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:05.986507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea89a69-a390-45c3-8760-65a494cbdbee None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:06.024425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e306c9-fc79-422e-ac37-8d961243b08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:06.025358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e306c9-fc79-422e-ac37-8d961243b08d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:06.025862 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e306c9-fc79-422e-ac37-8d961243b08d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:06.026312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e306c9-fc79-422e-ac37-8d961243b08d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:06.033799 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e306c9-fc79-422e-ac37-8d961243b08d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:06.035497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e306c9-fc79-422e-ac37-8d961243b08d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-44e306c9-fc79-422e-ac37-8d961243b08d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:25:06.047455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:06.048390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:06.049286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:25:06.049286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:25:06.049286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:25:06.049286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:25:06.049286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:25:06.049286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:25:06.049286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:06.049286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:06.049718 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 470/944] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:25:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:06.056100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009d2cb-96a9-401c-a83f-a6b8e8ea0e61 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:06.156910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009d2cb-96a9-401c-a83f-a6b8e8ea0e61 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:06.190908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85de6b-f55e-45d6-9ccd-68559965730c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:06.191571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85de6b-f55e-45d6-9ccd-68559965730c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:06.191844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85de6b-f55e-45d6-9ccd-68559965730c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:06.192110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85de6b-f55e-45d6-9ccd-68559965730c None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:06.192792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85de6b-f55e-45d6-9ccd-68559965730c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:06.193088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85de6b-f55e-45d6-9ccd-68559965730c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:06.194520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85de6b-f55e-45d6-9ccd-68559965730c None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:06.206980 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 475/945] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:25:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:06.212816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e306c9-fc79-422e-ac37-8d961243b08d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:06.220036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85de6b-f55e-45d6-9ccd-68559965730c None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:06.264317 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:06.297378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:06.339835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e306c9-fc79-422e-ac37-8d961243b08d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:06.344672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85de6b-f55e-45d6-9ccd-68559965730c None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:06.380634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bde61c-03d0-4e98-8b6c-9439e6ee4d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:06.381280 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bde61c-03d0-4e98-8b6c-9439e6ee4d0c None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:06.381551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bde61c-03d0-4e98-8b6c-9439e6ee4d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:06.381899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bde61c-03d0-4e98-8b6c-9439e6ee4d0c None admin] PATH_INFO: `/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/ba7a3df6fb6a4eb3a077a21b40794754/roles/6851de0f376f41a8a000506d1df67181` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:06.384880 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bde61c-03d0-4e98-8b6c-9439e6ee4d0c None admin] RBAC: Authorizing `identity:create_grant(project_id=29b2e27932984de79ee8dfbe95312ac2, user_id=ba7a3df6fb6a4eb3a077a21b40794754, role_id=6851de0f376f41a8a000506d1df67181)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:06.386592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bde61c-03d0-4e98-8b6c-9439e6ee4d0c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:06.390685 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53170324-ddf2-4e17-93d8-d62aa05dcb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:06.392013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53170324-ddf2-4e17-93d8-d62aa05dcb65 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:06.392486 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53170324-ddf2-4e17-93d8-d62aa05dcb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:06.392960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53170324-ddf2-4e17-93d8-d62aa05dcb65 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:06.393735 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53170324-ddf2-4e17-93d8-d62aa05dcb65 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:06.395293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53170324-ddf2-4e17-93d8-d62aa05dcb65 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-53170324-ddf2-4e17-93d8-d62aa05dcb65 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:06.404763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:06.411789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:06.412661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 476/946] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 16 05:25:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:06.414159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53170324-ddf2-4e17-93d8-d62aa05dcb65 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:06.465873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 471/947] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:25:06 2022] PUT /identity/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/ba7a3df6fb6a4eb3a077a21b40794754/roles/6851de0f376f41a8a000506d1df67181 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:25:06.539759 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53170324-ddf2-4e17-93d8-d62aa05dcb65 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:06.569800 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:06.582451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cbd6b4-6470-4eda-9218-49463d12f2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:06.583827 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbd6b4-6470-4eda-9218-49463d12f2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:06.584327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbd6b4-6470-4eda-9218-49463d12f2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:06.584788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cbd6b4-6470-4eda-9218-49463d12f2f6 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:06.585653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbd6b4-6470-4eda-9218-49463d12f2f6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:06.586164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbd6b4-6470-4eda-9218-49463d12f2f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:06.587870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cbd6b4-6470-4eda-9218-49463d12f2f6 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:06.594387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 477/948] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 16 05:25:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:06.601846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bde61c-03d0-4e98-8b6c-9439e6ee4d0c None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:06.718144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bde61c-03d0-4e98-8b6c-9439e6ee4d0c None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:06.720158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:06.753635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31b73f-ef6a-4216-a1f5-11e8c008e08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:06.754264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31b73f-ef6a-4216-a1f5-11e8c008e08d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:06.754538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31b73f-ef6a-4216-a1f5-11e8c008e08d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:06.754801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31b73f-ef6a-4216-a1f5-11e8c008e08d None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:06.755416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31b73f-ef6a-4216-a1f5-11e8c008e08d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:06.755708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31b73f-ef6a-4216-a1f5-11e8c008e08d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:06.757164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31b73f-ef6a-4216-a1f5-11e8c008e08d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:06.763418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 472/949] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 16 05:25:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:06.770264 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cbd6b4-6470-4eda-9218-49463d12f2f6 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:06.877000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cbd6b4-6470-4eda-9218-49463d12f2f6 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:06.915054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea88b9-d511-460c-b046-a9e0dbf817b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:06.915668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea88b9-d511-460c-b046-a9e0dbf817b2 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:06.915948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea88b9-d511-460c-b046-a9e0dbf817b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:06.916218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea88b9-d511-460c-b046-a9e0dbf817b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:06.916901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea88b9-d511-460c-b046-a9e0dbf817b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:06.918303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea88b9-d511-460c-b046-a9e0dbf817b2 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:07.042251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 478/950] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:25:06 2022] POST /identity/v3/endpoints => generated 349 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:25:07.807590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffda6f-ca70-4d5b-865e-fad789f99c7b None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:07.808432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffda6f-ca70-4d5b-865e-fad789f99c7b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:07.808853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffda6f-ca70-4d5b-865e-fad789f99c7b None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:07.810654 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 473/951] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:07.816265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6efd4-209b-42fa-a97a-9eb6dcc87cbb None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:07.816825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc85fc6-8431-4cc6-ac8c-d735c8f8a643 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:07.817254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc85fc6-8431-4cc6-ac8c-d735c8f8a643 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:07.817517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc85fc6-8431-4cc6-ac8c-d735c8f8a643 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:07.817703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6efd4-209b-42fa-a97a-9eb6dcc87cbb None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:07.818221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6efd4-209b-42fa-a97a-9eb6dcc87cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:07.819135 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 479/952] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:07.824376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7300f4-ab5d-4d7a-8e58-729aed1b7ced None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:07.824653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7300f4-ab5d-4d7a-8e58-729aed1b7ced None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:07.824923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7300f4-ab5d-4d7a-8e58-729aed1b7ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:07.839812 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-4ad6efd4-209b-42fa-a97a-9eb6dcc87cbb None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:07.841113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4ad6efd4-209b-42fa-a97a-9eb6dcc87cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:07.846548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-3e7300f4-ab5d-4d7a-8e58-729aed1b7ced None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:07.848101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3e7300f4-ab5d-4d7a-8e58-729aed1b7ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:07.896443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:07.941631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 474/953] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:07.945319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 480/954] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:08.212314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6efd4-209b-42fa-a97a-9eb6dcc87cbb None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:08.220970 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7300f4-ab5d-4d7a-8e58-729aed1b7ced None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:08.325854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:08.327856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7300f4-ab5d-4d7a-8e58-729aed1b7ced None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:08.336082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6efd4-209b-42fa-a97a-9eb6dcc87cbb None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:08.363424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310c7eb-c480-4d8f-b9c0-76dbed6db926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:08.364124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310c7eb-c480-4d8f-b9c0-76dbed6db926 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:08.364460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310c7eb-c480-4d8f-b9c0-76dbed6db926 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:08.364788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310c7eb-c480-4d8f-b9c0-76dbed6db926 None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:08.365487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310c7eb-c480-4d8f-b9c0-76dbed6db926 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:08.367002 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310c7eb-c480-4d8f-b9c0-76dbed6db926 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:08.370854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: WARNING keystone.server.flask.common [None req-3310c7eb-c480-4d8f-b9c0-76dbed6db926 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 16 05:25:08.378297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b876076-6dee-4e33-a131-f66d14ca0ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:08.379006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b876076-6dee-4e33-a131-f66d14ca0ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:08.379346 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b876076-6dee-4e33-a131-f66d14ca0ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:08.379677 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b876076-6dee-4e33-a131-f66d14ca0ae0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:08.380752 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b876076-6dee-4e33-a131-f66d14ca0ae0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:08.382575 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b876076-6dee-4e33-a131-f66d14ca0ae0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-1b876076-6dee-4e33-a131-f66d14ca0ae0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:08.393121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:08.394428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:08.395361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 475/955] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:25:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:08.399998 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b876076-6dee-4e33-a131-f66d14ca0ae0 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:08.421485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.sql.core [None req-3310c7eb-c480-4d8f-b9c0-76dbed6db926 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 16 05:25:08.421485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 16 05:25:08.421485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [parameters: {'id': '38fc2580da194dccbbf2bb006d39a175', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 16 05:25:08.421485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81557) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 16 05:25:08.445955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-3310c7eb-c480-4d8f-b9c0-76dbed6db926 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 38fc2580da194dccbbf2bb006d39a175.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 38fc2580da194dccbbf2bb006d39a175. Sep 16 05:25:08.445955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:08.445955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 05:25:08.445955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 05:25:08.445955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 05:25:08.445955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 05:25:08.445955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 05:25:08.445955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 05:25:08.445955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 05:25:08.445955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application conn.query(q) Sep 16 05:25:08.445955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 05:25:08.445955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 05:25:08.445955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application result.read() Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 16 05:25:08.446996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application next(self.gen) Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application yield resource Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application next(self.gen) Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self.session.rollback() Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self.force_reraise() Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise self.value Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application session.commit() Sep 16 05:25:08.448091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._prepare_impl() Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self.session.flush() Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._flush(objects) Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application compat.raise_( Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application flush_context.execute() Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 16 05:25:08.449005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rec.execute(self) Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 16 05:25:08.450688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application conn.query(q) Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application result.read() Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 05:25:08.453318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [parameters: {'id': '38fc2580da194dccbbf2bb006d39a175', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:08.456408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:08.458536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:08.461168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:08.461168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 16 05:25:08.461168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 16 05:25:08.461168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:08.461168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:08.461168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 16 05:25:08.461168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 16 05:25:08.461168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 38fc2580da194dccbbf2bb006d39a175. Sep 16 05:25:08.461168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:08.461168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 481/956] 10.222.0.49 () {60 vars in 1140 bytes} [Fri Sep 16 05:25:08 2022] POST /identity/v3/projects => generated 262 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:08.461168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310c7eb-c480-4d8f-b9c0-76dbed6db926 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:08.497610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:08.523792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b876076-6dee-4e33-a131-f66d14ca0ae0 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:08.561246 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e55427-b477-4b0c-b710-220b95754915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:08.561777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e55427-b477-4b0c-b710-220b95754915 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:08.562125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e55427-b477-4b0c-b710-220b95754915 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:08.562398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e55427-b477-4b0c-b710-220b95754915 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:08.563479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e55427-b477-4b0c-b710-220b95754915 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:08.563991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e55427-b477-4b0c-b710-220b95754915 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:08.566888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e55427-b477-4b0c-b710-220b95754915 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:08.577081 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 476/957] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 16 05:25:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:08.577975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e55427-b477-4b0c-b710-220b95754915 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:08.599476 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310c7eb-c480-4d8f-b9c0-76dbed6db926 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:08.602567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:08.645976 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb9743-3f0b-4e93-96e3-4ac3380f38f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:08.647698 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb9743-3f0b-4e93-96e3-4ac3380f38f7 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:08.648176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb9743-3f0b-4e93-96e3-4ac3380f38f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:08.648889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb9743-3f0b-4e93-96e3-4ac3380f38f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:08.659911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb9743-3f0b-4e93-96e3-4ac3380f38f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:08.662396 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb9743-3f0b-4e93-96e3-4ac3380f38f7 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-0eeb9743-3f0b-4e93-96e3-4ac3380f38f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:25:08.677348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:08.678347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:08.679997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:25:08.679997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:25:08.679997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:25:08.679997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:25:08.679997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:25:08.679997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:25:08.679997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:08.679997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:08.683402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 482/958] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:25:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:08.695662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb9743-3f0b-4e93-96e3-4ac3380f38f7 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:08.705294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e55427-b477-4b0c-b710-220b95754915 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:08.744097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49be0fe-eb51-4a64-ac01-bacaa50df113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:08.744701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49be0fe-eb51-4a64-ac01-bacaa50df113 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:08.744977 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49be0fe-eb51-4a64-ac01-bacaa50df113 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:08.745245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49be0fe-eb51-4a64-ac01-bacaa50df113 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:08.745913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49be0fe-eb51-4a64-ac01-bacaa50df113 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:08.746218 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49be0fe-eb51-4a64-ac01-bacaa50df113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:08.747686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49be0fe-eb51-4a64-ac01-bacaa50df113 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:08.752905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 477/959] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 16 05:25:08 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:08.807999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de8556-b1f8-445e-9e6a-d8c9cc6bb8b1 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:08.808653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de8556-b1f8-445e-9e6a-d8c9cc6bb8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:08.808653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de8556-b1f8-445e-9e6a-d8c9cc6bb8b1 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:08.810216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 478/960] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:25:08.811595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:08.816671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6e54e-f6dd-42c1-817e-a1d7b26c2d94 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:08.816947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6e54e-f6dd-42c1-817e-a1d7b26c2d94 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:08.817209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6e54e-f6dd-42c1-817e-a1d7b26c2d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:08.824666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb9743-3f0b-4e93-96e3-4ac3380f38f7 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:08.839076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f2a6e54e-f6dd-42c1-817e-a1d7b26c2d94 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:08.841393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f2a6e54e-f6dd-42c1-817e-a1d7b26c2d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:08.871223 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fd494-fb97-4bea-8cf5-899ad2ccb614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:08.871838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fd494-fb97-4bea-8cf5-899ad2ccb614 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:08.872112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fd494-fb97-4bea-8cf5-899ad2ccb614 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:08.872376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fd494-fb97-4bea-8cf5-899ad2ccb614 None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:08.873936 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fd494-fb97-4bea-8cf5-899ad2ccb614 None admin] RBAC: Adding query filter params (name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:08.874273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fd494-fb97-4bea-8cf5-899ad2ccb614 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:08.875677 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fd494-fb97-4bea-8cf5-899ad2ccb614 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:08.898155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 483/961] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 16 05:25:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:08.933900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 479/962] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:09.200372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fd494-fb97-4bea-8cf5-899ad2ccb614 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:09.316880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fd494-fb97-4bea-8cf5-899ad2ccb614 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:09.354287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e967bd7-67bf-474f-bde3-fb4eeeaa60ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:09.354917 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e967bd7-67bf-474f-bde3-fb4eeeaa60ef None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:09.355187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e967bd7-67bf-474f-bde3-fb4eeeaa60ef None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:09.355450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e967bd7-67bf-474f-bde3-fb4eeeaa60ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:09.358110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13cd5b-ba12-4ede-babb-4503025e4e12 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:09.358404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13cd5b-ba12-4ede-babb-4503025e4e12 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:09.358678 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13cd5b-ba12-4ede-babb-4503025e4e12 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:09.360271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 480/963] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:09.365282 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c13459-83b1-4da5-b892-cd9cf1136ed9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:09.365556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c13459-83b1-4da5-b892-cd9cf1136ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:09.365842 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c13459-83b1-4da5-b892-cd9cf1136ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:09.367340 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e967bd7-67bf-474f-bde3-fb4eeeaa60ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-0e967bd7-67bf-474f-bde3-fb4eeeaa60ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:25:09.372243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:09.373373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:09.374405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:09.374405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:09.374405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:09.374405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:09.374405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:25:09.374405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:25:09.374405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:25:09.374405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:25:09.374405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:25:09.374405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:25:09.374405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 05:25:09.374405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:09.374405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 484/964] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Sep 16 05:25:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:09.381057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e967bd7-67bf-474f-bde3-fb4eeeaa60ef None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:09.384771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-09c13459-83b1-4da5-b892-cd9cf1136ed9 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:09.385981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-09c13459-83b1-4da5-b892-cd9cf1136ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:09.392097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:09.396686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:09.462162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 481/965] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:09.490339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e967bd7-67bf-474f-bde3-fb4eeeaa60ef None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:09.528891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a92c6-3433-4e44-8d9b-ba92da7eb04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:09.530094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a92c6-3433-4e44-8d9b-ba92da7eb04e None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:09.530629 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a92c6-3433-4e44-8d9b-ba92da7eb04e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:09.531110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a92c6-3433-4e44-8d9b-ba92da7eb04e None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:09.531998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a92c6-3433-4e44-8d9b-ba92da7eb04e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:09.532490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a92c6-3433-4e44-8d9b-ba92da7eb04e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:09.534110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67a92c6-3433-4e44-8d9b-ba92da7eb04e None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:09.543775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 485/966] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 16 05:25:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:09.550426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c13459-83b1-4da5-b892-cd9cf1136ed9 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:09.660317 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c13459-83b1-4da5-b892-cd9cf1136ed9 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:09.695567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac9420-cbbf-471f-9ad7-7ca464afe0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:09.696199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac9420-cbbf-471f-9ad7-7ca464afe0cf None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:09.696471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac9420-cbbf-471f-9ad7-7ca464afe0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:09.696739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac9420-cbbf-471f-9ad7-7ca464afe0cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:09.707816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac9420-cbbf-471f-9ad7-7ca464afe0cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:09.709254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac9420-cbbf-471f-9ad7-7ca464afe0cf None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:09.711894 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-1dac9420-cbbf-471f-9ad7-7ca464afe0cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:25:09.725600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:09.726618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:25:09.727520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:25:09.727520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:09.727520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:09.727520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 482/967] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:09.734665 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a92c6-3433-4e44-8d9b-ba92da7eb04e None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:09.766382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac9420-cbbf-471f-9ad7-7ca464afe0cf None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:09.810366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:09.849113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a92c6-3433-4e44-8d9b-ba92da7eb04e None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:09.889181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678d7a7-dca3-4df7-a587-b6602037e75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:09.889649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac9420-cbbf-471f-9ad7-7ca464afe0cf None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:09.890739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678d7a7-dca3-4df7-a587-b6602037e75a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:09.891297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678d7a7-dca3-4df7-a587-b6602037e75a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:09.892041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3678d7a7-dca3-4df7-a587-b6602037e75a None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:09.892992 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678d7a7-dca3-4df7-a587-b6602037e75a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:09.893574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678d7a7-dca3-4df7-a587-b6602037e75a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:09.895248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3678d7a7-dca3-4df7-a587-b6602037e75a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:09.914211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 486/968] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:09.915450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678d7a7-dca3-4df7-a587-b6602037e75a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:09.930117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37155e14-b133-4bc3-b7b7-a477ef868846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:09.931005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37155e14-b133-4bc3-b7b7-a477ef868846 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:09.931005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37155e14-b133-4bc3-b7b7-a477ef868846 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:09.931263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37155e14-b133-4bc3-b7b7-a477ef868846 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:09.931855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37155e14-b133-4bc3-b7b7-a477ef868846 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:09.933205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37155e14-b133-4bc3-b7b7-a477ef868846 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-37155e14-b133-4bc3-b7b7-a477ef868846 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:09.942780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:09.943816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:25:09.944705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:09.948857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 483/969] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 16 05:25:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:09.949777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37155e14-b133-4bc3-b7b7-a477ef868846 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:10.012117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:10.031302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3678d7a7-dca3-4df7-a587-b6602037e75a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:10.054857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37155e14-b133-4bc3-b7b7-a477ef868846 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:10.060042 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:10.068961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c396e-e6a3-44e2-a0bb-b3eeceda2b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:10.069701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c396e-e6a3-44e2-a0bb-b3eeceda2b6b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:10.070049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c396e-e6a3-44e2-a0bb-b3eeceda2b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:10.070370 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7c396e-e6a3-44e2-a0bb-b3eeceda2b6b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:10.078704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c396e-e6a3-44e2-a0bb-b3eeceda2b6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:10.080264 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7c396e-e6a3-44e2-a0bb-b3eeceda2b6b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:10.093686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb1772-4c31-499b-bfae-1016ccfcb004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-6b7c396e-e6a3-44e2-a0bb-b3eeceda2b6b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:25:10.094120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:10.095279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:10.096473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:10.096473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:10.096473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:25:10.096473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:25:10.096473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:25:10.096473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:25:10.096473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:25:10.096473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:25:10.096473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 16 05:25:10.096473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:10.097986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb1772-4c31-499b-bfae-1016ccfcb004 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:10.098457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb1772-4c31-499b-bfae-1016ccfcb004 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:10.099105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb1772-4c31-499b-bfae-1016ccfcb004 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:10.099924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb1772-4c31-499b-bfae-1016ccfcb004 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:10.100365 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb1772-4c31-499b-bfae-1016ccfcb004 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:10.101940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb1772-4c31-499b-bfae-1016ccfcb004 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:10.103835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 487/970] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 16 05:25:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:10.105182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c396e-e6a3-44e2-a0bb-b3eeceda2b6b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:10.113681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 484/971] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 16 05:25:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:10.114798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb1772-4c31-499b-bfae-1016ccfcb004 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:10.214179 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7c396e-e6a3-44e2-a0bb-b3eeceda2b6b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:10.227534 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb1772-4c31-499b-bfae-1016ccfcb004 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:10.250220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1c448-2882-4063-8363-e1acfb0445cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:10.251048 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1c448-2882-4063-8363-e1acfb0445cd None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:10.251416 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1c448-2882-4063-8363-e1acfb0445cd None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:10.251781 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1c448-2882-4063-8363-e1acfb0445cd None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:10.252644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1c448-2882-4063-8363-e1acfb0445cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:10.252999 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1c448-2882-4063-8363-e1acfb0445cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:10.254639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b1c448-2882-4063-8363-e1acfb0445cd None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:10.255201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:10.269922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9be5f-e642-468b-b5f9-363524b23ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:10.269922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9be5f-e642-468b-b5f9-363524b23ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:10.269922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9be5f-e642-468b-b5f9-363524b23ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:10.269922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9be5f-e642-468b-b5f9-363524b23ca1 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:10.270374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9be5f-e642-468b-b5f9-363524b23ca1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:10.270374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9be5f-e642-468b-b5f9-363524b23ca1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:10.270931 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9be5f-e642-468b-b5f9-363524b23ca1 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:10.281981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 485/973] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 16 05:25:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:10.282170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9be5f-e642-468b-b5f9-363524b23ca1 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:10.282485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 488/973] 10.222.0.49 () {56 vars in 1149 bytes} [Fri Sep 16 05:25:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:10.283745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1c448-2882-4063-8363-e1acfb0445cd None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:10.333970 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:10.390211 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9be5f-e642-468b-b5f9-363524b23ca1 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:10.390645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1c448-2882-4063-8363-e1acfb0445cd None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:10.427392 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c36a7a-8d26-4773-8526-4b6d7ff00710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:10.428045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c36a7a-8d26-4773-8526-4b6d7ff00710 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:10.428320 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c36a7a-8d26-4773-8526-4b6d7ff00710 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:10.428587 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c36a7a-8d26-4773-8526-4b6d7ff00710 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:10.428906 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8bf85-d79d-4681-b1ce-ea85e67a7a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:10.429350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c36a7a-8d26-4773-8526-4b6d7ff00710 None admin] RBAC: Adding query filter params (interface=admin, service_id=37f365e213a44e72a65c4cd11ba1543f, region_id=RegionOne) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:10.429652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c36a7a-8d26-4773-8526-4b6d7ff00710 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:10.430130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8bf85-d79d-4681-b1ce-ea85e67a7a97 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:10.430592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8bf85-d79d-4681-b1ce-ea85e67a7a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:10.431061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c36a7a-8d26-4773-8526-4b6d7ff00710 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:10.431270 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8bf85-d79d-4681-b1ce-ea85e67a7a97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:10.441501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 486/974] 10.222.0.49 () {56 vars in 1249 bytes} [Fri Sep 16 05:25:10 2022] GET /identity/v3/endpoints?service_id=37f365e213a44e72a65c4cd11ba1543f&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:10.442407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c36a7a-8d26-4773-8526-4b6d7ff00710 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:10.499145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:10.504783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8bf85-d79d-4681-b1ce-ea85e67a7a97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:10.506480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8bf85-d79d-4681-b1ce-ea85e67a7a97 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:10.549452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c36a7a-8d26-4773-8526-4b6d7ff00710 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:10.560947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:10.587917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd062a4-ea46-4cec-986d-9f75b05ff1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:10.588894 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd062a4-ea46-4cec-986d-9f75b05ff1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:10.589400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd062a4-ea46-4cec-986d-9f75b05ff1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:10.589908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd062a4-ea46-4cec-986d-9f75b05ff1a5 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:10.590711 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd062a4-ea46-4cec-986d-9f75b05ff1a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:10.592818 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd062a4-ea46-4cec-986d-9f75b05ff1a5 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:10.606332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 487/975] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 16 05:25:10 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:25:10.619416 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 489/976] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:25:10 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:25:10.625884 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd062a4-ea46-4cec-986d-9f75b05ff1a5 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:10.750693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd062a4-ea46-4cec-986d-9f75b05ff1a5 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:10.770537 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:10.789834 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b4461-656b-4234-98d9-6f1f808077b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:10.790493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b4461-656b-4234-98d9-6f1f808077b5 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:10.790769 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b4461-656b-4234-98d9-6f1f808077b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:10.791061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b4461-656b-4234-98d9-6f1f808077b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:10.802082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b4461-656b-4234-98d9-6f1f808077b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:10.803467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b4461-656b-4234-98d9-6f1f808077b5 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-ee7b4461-656b-4234-98d9-6f1f808077b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:25:10.819446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:10.820430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:25:10.821337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:25:10.821337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:10.821337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:10.821337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 488/977] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:10.828487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8bf85-d79d-4681-b1ce-ea85e67a7a97 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:10.940193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8bf85-d79d-4681-b1ce-ea85e67a7a97 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:10.980824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5fbc86-ff40-4fa4-b179-b794e49f4c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:10.981460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fbc86-ff40-4fa4-b179-b794e49f4c68 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:10.981736 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fbc86-ff40-4fa4-b179-b794e49f4c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:10.982031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fbc86-ff40-4fa4-b179-b794e49f4c68 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:10.983628 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fbc86-ff40-4fa4-b179-b794e49f4c68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:10.983933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fbc86-ff40-4fa4-b179-b794e49f4c68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:10.985318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5fbc86-ff40-4fa4-b179-b794e49f4c68 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:10.999382 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 490/978] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:11.004946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b4461-656b-4234-98d9-6f1f808077b5 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:11.077625 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713e05a-ddc3-414e-bc78-b359211ac11a None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:11.077809 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713e05a-ddc3-414e-bc78-b359211ac11a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:11.078130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d713e05a-ddc3-414e-bc78-b359211ac11a None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:11.079689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 491/979] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:11.085586 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b4948-5416-442f-9bb6-c21c0c2cdb9e None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:11.086248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b4948-5416-442f-9bb6-c21c0c2cdb9e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:11.086716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b4948-5416-442f-9bb6-c21c0c2cdb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:11.101458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:11.108347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-077b4948-5416-442f-9bb6-c21c0c2cdb9e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:11.110255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-077b4948-5416-442f-9bb6-c21c0c2cdb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:11.124336 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b4461-656b-4234-98d9-6f1f808077b5 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:11.159657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba3902-f153-4533-acff-576aac5cbf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:11.161055 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba3902-f153-4533-acff-576aac5cbf05 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:11.161550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba3902-f153-4533-acff-576aac5cbf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:11.162045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba3902-f153-4533-acff-576aac5cbf05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:11.172871 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba3902-f153-4533-acff-576aac5cbf05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:11.174730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba3902-f153-4533-acff-576aac5cbf05 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-e6ba3902-f153-4533-acff-576aac5cbf05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:25:11.186993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:11.187952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:11.188857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:25:11.188857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:25:11.188857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:25:11.188857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:25:11.188857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:25:11.188857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:25:11.188857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:11.188857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:11.191245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 489/980] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:25:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:11.194262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8334f-291c-4b1d-8197-9c04eaca3bb9 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:11.194544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8334f-291c-4b1d-8197-9c04eaca3bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:11.194815 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8334f-291c-4b1d-8197-9c04eaca3bb9 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:11.196213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 490/981] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:11.196952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8334f-291c-4b1d-8197-9c04eaca3bb9 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:11.206148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 492/982] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:11.207479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0461bdf-64c2-47e6-a1d1-f947975877ed None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:11.207762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0461bdf-64c2-47e6-a1d1-f947975877ed None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:11.208030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0461bdf-64c2-47e6-a1d1-f947975877ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:11.226115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a0461bdf-64c2-47e6-a1d1-f947975877ed None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:11.227300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a0461bdf-64c2-47e6-a1d1-f947975877ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:11.236777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:11.313849 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 493/983] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:11.335324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8334f-291c-4b1d-8197-9c04eaca3bb9 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:11.371328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62971e7e-65a0-4f8c-ae36-b504f68c5505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:11.371924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62971e7e-65a0-4f8c-ae36-b504f68c5505 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:11.372196 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62971e7e-65a0-4f8c-ae36-b504f68c5505 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:11.372457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62971e7e-65a0-4f8c-ae36-b504f68c5505 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:11.373292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62971e7e-65a0-4f8c-ae36-b504f68c5505 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:11.373776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62971e7e-65a0-4f8c-ae36-b504f68c5505 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:11.375242 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62971e7e-65a0-4f8c-ae36-b504f68c5505 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:11.387907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 491/984] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:25:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:11.394404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0461bdf-64c2-47e6-a1d1-f947975877ed None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:11.462429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:11.484675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62971e7e-65a0-4f8c-ae36-b504f68c5505 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:11.502155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0461bdf-64c2-47e6-a1d1-f947975877ed None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:11.545794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cd20c-5ab0-44e1-b5e8-9f8c4f35faea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:11.546527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cd20c-5ab0-44e1-b5e8-9f8c4f35faea None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:11.546859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cd20c-5ab0-44e1-b5e8-9f8c4f35faea None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:11.547193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cd20c-5ab0-44e1-b5e8-9f8c4f35faea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:11.547970 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cd20c-5ab0-44e1-b5e8-9f8c4f35faea None admin] RBAC: Adding query filter params (role.id=6851de0f376f41a8a000506d1df67181, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=ba7a3df6fb6a4eb3a077a21b40794754) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:11.548345 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cd20c-5ab0-44e1-b5e8-9f8c4f35faea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:11.549807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cd20c-5ab0-44e1-b5e8-9f8c4f35faea None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:11.556267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 494/985] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:25:11 2022] GET /identity/v3/role_assignments?user.id=ba7a3df6fb6a4eb3a077a21b40794754&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=6851de0f376f41a8a000506d1df67181 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:11.596792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:11.611155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62971e7e-65a0-4f8c-ae36-b504f68c5505 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:11.621276 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cd20c-5ab0-44e1-b5e8-9f8c4f35faea None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:11.657946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e7ad9-abb5-437d-baa4-d39fd23fe4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:11.670541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e7ad9-abb5-437d-baa4-d39fd23fe4e6 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:11.670541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e7ad9-abb5-437d-baa4-d39fd23fe4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:11.670541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e7ad9-abb5-437d-baa4-d39fd23fe4e6 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:11.670541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e7ad9-abb5-437d-baa4-d39fd23fe4e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:11.670541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e7ad9-abb5-437d-baa4-d39fd23fe4e6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:11.736950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cd20c-5ab0-44e1-b5e8-9f8c4f35faea None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:11.773386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:11.783750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 492/986] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:25:11 2022] POST /identity/v3/services => generated 242 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:25:11.785232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059e948-b32e-40ed-8d2d-234ba97d8ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:11.787961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059e948-b32e-40ed-8d2d-234ba97d8ce1 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:11.787961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059e948-b32e-40ed-8d2d-234ba97d8ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:11.787961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059e948-b32e-40ed-8d2d-234ba97d8ce1 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:11.787961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059e948-b32e-40ed-8d2d-234ba97d8ce1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:11.788823 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059e948-b32e-40ed-8d2d-234ba97d8ce1 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:11.844066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.sql.core [None req-6059e948-b32e-40ed-8d2d-234ba97d8ce1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 16 05:25:11.844066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 16 05:25:11.844066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [parameters: {'user_id': '271418d443824c1daec828d99d05d42f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 16 05:25:11.844066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81557) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-6059e948-b32e-40ed-8d2d-234ba97d8ce1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application conn.query(q) Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application result.read() Sep 16 05:25:11.852445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 05:25:11.853367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application next(self.gen) Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application yield resource Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application next(self.gen) Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self.session.rollback() Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self.force_reraise() Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise self.value Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application session.commit() Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 16 05:25:11.854652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._prepare_impl() Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self.session.flush() Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._flush(objects) Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application compat.raise_( Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application flush_context.execute() Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rec.execute(self) Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 16 05:25:11.855741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 16 05:25:11.856847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application result = self._query(query) Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application conn.query(q) Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application result.read() Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 16 05:25:11.858931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [parameters: {'user_id': '271418d443824c1daec828d99d05d42f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:11.859986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 16 05:25:11.860982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 16 05:25:11.862438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 16 05:25:11.862438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 16 05:25:11.862438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 16 05:25:11.862438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 16 05:25:11.862438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 16 05:25:11.862438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:11.862438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 495/987] 10.222.0.49 () {60 vars in 1131 bytes} [Fri Sep 16 05:25:11 2022] POST /identity/v3/users => generated 160 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:11.868232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e7ad9-abb5-437d-baa4-d39fd23fe4e6 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:12.003759 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e7ad9-abb5-437d-baa4-d39fd23fe4e6 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:12.039685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdc0bf-fa03-4ca8-960b-aef46469551a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:12.040474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdc0bf-fa03-4ca8-960b-aef46469551a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:12.040644 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdc0bf-fa03-4ca8-960b-aef46469551a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:12.040903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdc0bf-fa03-4ca8-960b-aef46469551a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:12.049564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdc0bf-fa03-4ca8-960b-aef46469551a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:12.051066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdc0bf-fa03-4ca8-960b-aef46469551a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-cfbdc0bf-fa03-4ca8-960b-aef46469551a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:25:12.062647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:12.063612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:12.064583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:12.064583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:12.064583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:25:12.064583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:25:12.064583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:25:12.064583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:25:12.064583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:25:12.064583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:25:12.064583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 16 05:25:12.064583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:12.064583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 493/988] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:25:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:12.070225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059e948-b32e-40ed-8d2d-234ba97d8ce1 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:12.175725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059e948-b32e-40ed-8d2d-234ba97d8ce1 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:12.212895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fa9d2c-f6d7-489b-b1e5-e1fa3e05a0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:12.213744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa9d2c-f6d7-489b-b1e5-e1fa3e05a0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:12.214188 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa9d2c-f6d7-489b-b1e5-e1fa3e05a0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:12.214602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fa9d2c-f6d7-489b-b1e5-e1fa3e05a0b5 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:12.215460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa9d2c-f6d7-489b-b1e5-e1fa3e05a0b5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:12.215922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa9d2c-f6d7-489b-b1e5-e1fa3e05a0b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:12.218119 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fa9d2c-f6d7-489b-b1e5-e1fa3e05a0b5 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:12.257376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 496/989] 10.222.0.49 () {56 vars in 1101 bytes} [Fri Sep 16 05:25:12 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:12.883645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80015a67-d2a7-4737-b409-a5b325a6794a None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:12.884023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80015a67-d2a7-4737-b409-a5b325a6794a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:12.884225 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80015a67-d2a7-4737-b409-a5b325a6794a None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:12.885535 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 494/990] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:12.891425 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fa49b-3549-4937-9c75-1ba562c15e2c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:12.891961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fa49b-3549-4937-9c75-1ba562c15e2c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:12.892232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fa49b-3549-4937-9c75-1ba562c15e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:12.913273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-cf1fa49b-3549-4937-9c75-1ba562c15e2c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:12.913273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-cf1fa49b-3549-4937-9c75-1ba562c15e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:12.958382 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:12.985903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 497/991] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:13.250471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80015a67-d2a7-4737-b409-a5b325a6794a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:13.262039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:13.353791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80015a67-d2a7-4737-b409-a5b325a6794a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:13.387433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65036500-e039-4174-a191-a18bd24e8c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:13.388054 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65036500-e039-4174-a191-a18bd24e8c40 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:13.388322 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65036500-e039-4174-a191-a18bd24e8c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:13.388689 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65036500-e039-4174-a191-a18bd24e8c40 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:13.389341 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65036500-e039-4174-a191-a18bd24e8c40 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:13.390740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65036500-e039-4174-a191-a18bd24e8c40 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-65036500-e039-4174-a191-a18bd24e8c40 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:13.399631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:13.400625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 16 05:25:13.401474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:13.403019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 495/992] 10.222.0.49 () {56 vars in 1126 bytes} [Fri Sep 16 05:25:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:13.407429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fa49b-3549-4937-9c75-1ba562c15e2c None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:13.505106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fa49b-3549-4937-9c75-1ba562c15e2c None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:13.542885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0da9fa-7f41-4e4a-9ad9-914aa2b4f1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:13.543486 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0da9fa-7f41-4e4a-9ad9-914aa2b4f1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:13.543754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0da9fa-7f41-4e4a-9ad9-914aa2b4f1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:13.544015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0da9fa-7f41-4e4a-9ad9-914aa2b4f1e8 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:13.544636 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0da9fa-7f41-4e4a-9ad9-914aa2b4f1e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:13.545522 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0da9fa-7f41-4e4a-9ad9-914aa2b4f1e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:13.546972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0da9fa-7f41-4e4a-9ad9-914aa2b4f1e8 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:13.552037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 498/993] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 16 05:25:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:13.557468 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65036500-e039-4174-a191-a18bd24e8c40 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:13.627512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:13.657330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65036500-e039-4174-a191-a18bd24e8c40 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:13.691065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f63a6-247f-4bc5-b406-b0e37cec75d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:13.691663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f63a6-247f-4bc5-b406-b0e37cec75d1 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:13.691929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f63a6-247f-4bc5-b406-b0e37cec75d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:13.692194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f63a6-247f-4bc5-b406-b0e37cec75d1 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:13.692808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f63a6-247f-4bc5-b406-b0e37cec75d1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:13.693177 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f63a6-247f-4bc5-b406-b0e37cec75d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:13.694497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f63a6-247f-4bc5-b406-b0e37cec75d1 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:13.699399 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 496/994] 10.222.0.49 () {56 vars in 1122 bytes} [Fri Sep 16 05:25:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:13.705535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0da9fa-7f41-4e4a-9ad9-914aa2b4f1e8 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:13.748851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:13.812254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0da9fa-7f41-4e4a-9ad9-914aa2b4f1e8 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:13.846408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adae6d-c5aa-42bb-b929-7dde529d5446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:13.847005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adae6d-c5aa-42bb-b929-7dde529d5446 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:13.847272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adae6d-c5aa-42bb-b929-7dde529d5446 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:13.847531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adae6d-c5aa-42bb-b929-7dde529d5446 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:13.848192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adae6d-c5aa-42bb-b929-7dde529d5446 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:13.849531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47adae6d-c5aa-42bb-b929-7dde529d5446 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:13.910940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a1378-4b7c-4e8a-b1f0-26fac383fdb2 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:13.911274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a1378-4b7c-4e8a-b1f0-26fac383fdb2 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:13.911442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a1378-4b7c-4e8a-b1f0-26fac383fdb2 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:13.913375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 497/995] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:25:13.919060 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0368c5f-19d1-4ae4-9568-10f34ebdd495 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:13.919368 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0368c5f-19d1-4ae4-9568-10f34ebdd495 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:13.919589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0368c5f-19d1-4ae4-9568-10f34ebdd495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:13.929891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 499/996] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:25:13 2022] POST /identity/v3/endpoints => generated 325 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:25:13.940864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f0368c5f-19d1-4ae4-9568-10f34ebdd495 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:13.942126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f0368c5f-19d1-4ae4-9568-10f34ebdd495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:14.022398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 498/997] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:14.026054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fa96a-da45-49e2-9aa1-130213899c15 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:14.026338 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fa96a-da45-49e2-9aa1-130213899c15 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:14.026629 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fa96a-da45-49e2-9aa1-130213899c15 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:14.028093 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 500/998] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:14.033681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dc900-13e8-4102-8c37-d0ee111f4db4 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:14.033995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dc900-13e8-4102-8c37-d0ee111f4db4 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:14.034264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7dc900-13e8-4102-8c37-d0ee111f4db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:14.053797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0f7dc900-13e8-4102-8c37-d0ee111f4db4 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:14.055044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0f7dc900-13e8-4102-8c37-d0ee111f4db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:14.117768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:14.129071 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 499/999] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:14.288520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fa96a-da45-49e2-9aa1-130213899c15 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:14.386155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:14.391295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fa96a-da45-49e2-9aa1-130213899c15 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:14.400774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7dc900-13e8-4102-8c37-d0ee111f4db4 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:14.425072 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d687f83-205a-46ce-9587-682f0f979938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:14.425751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d687f83-205a-46ce-9587-682f0f979938 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:14.426187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d687f83-205a-46ce-9587-682f0f979938 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:14.426678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d687f83-205a-46ce-9587-682f0f979938 None admin] PATH_INFO: `/v3/services/image` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:14.427307 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d687f83-205a-46ce-9587-682f0f979938 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:14.429599 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d687f83-205a-46ce-9587-682f0f979938 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-2d687f83-205a-46ce-9587-682f0f979938 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:14.439181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:14.442867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:14.443708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 501/1000] 10.222.0.49 () {56 vars in 1105 bytes} [Fri Sep 16 05:25:14 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:14.446519 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d687f83-205a-46ce-9587-682f0f979938 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:14.506986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7dc900-13e8-4102-8c37-d0ee111f4db4 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:14.543397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a9fe8-d307-4fbb-8558-0ab312b41d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:14.544195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a9fe8-d307-4fbb-8558-0ab312b41d9e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:14.544473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a9fe8-d307-4fbb-8558-0ab312b41d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:14.545281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a9fe8-d307-4fbb-8558-0ab312b41d9e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:14.545885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a9fe8-d307-4fbb-8558-0ab312b41d9e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:14.547698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a9fe8-d307-4fbb-8558-0ab312b41d9e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:14.555200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d687f83-205a-46ce-9587-682f0f979938 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-2a9a9fe8-d307-4fbb-8558-0ab312b41d9e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:14.556713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:14.557704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:14.558597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 500/1001] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:25:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:14.563664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a9fe8-d307-4fbb-8558-0ab312b41d9e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:14.592963 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c8269-aba3-4af7-a9ca-9ef87a2eba2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:14.593547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c8269-aba3-4af7-a9ca-9ef87a2eba2b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:14.593840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c8269-aba3-4af7-a9ca-9ef87a2eba2b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:14.594203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c8269-aba3-4af7-a9ca-9ef87a2eba2b None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:14.594714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c8269-aba3-4af7-a9ca-9ef87a2eba2b None admin] RBAC: Adding query filter params (name=image) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:14.595004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c8269-aba3-4af7-a9ca-9ef87a2eba2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:14.596342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c8269-aba3-4af7-a9ca-9ef87a2eba2b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:14.600935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 502/1002] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:14.606426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c8269-aba3-4af7-a9ca-9ef87a2eba2b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:14.626245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:14.642637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:14.673147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a9fe8-d307-4fbb-8558-0ab312b41d9e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:14.710933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae63aa8-1688-4a98-9520-296b422c7c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:14.711532 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae63aa8-1688-4a98-9520-296b422c7c8d None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:14.711797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae63aa8-1688-4a98-9520-296b422c7c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:14.712070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae63aa8-1688-4a98-9520-296b422c7c8d None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:14.712692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae63aa8-1688-4a98-9520-296b422c7c8d None admin] RBAC: Adding query filter params (name=compute) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:14.712988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae63aa8-1688-4a98-9520-296b422c7c8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:14.714376 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae63aa8-1688-4a98-9520-296b422c7c8d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:14.719852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c8269-aba3-4af7-a9ca-9ef87a2eba2b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:14.721064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 501/1003] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 16 05:25:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:14.726538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae63aa8-1688-4a98-9520-296b422c7c8d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:14.762487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86febe1-7a44-4a83-8f56-03972437113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:14.763090 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86febe1-7a44-4a83-8f56-03972437113f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:14.763363 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86febe1-7a44-4a83-8f56-03972437113f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:14.764853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86febe1-7a44-4a83-8f56-03972437113f None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:14.765486 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86febe1-7a44-4a83-8f56-03972437113f None admin] RBAC: Adding query filter params (type=image) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:14.765785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86febe1-7a44-4a83-8f56-03972437113f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:14.767432 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86febe1-7a44-4a83-8f56-03972437113f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:14.771933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 503/1004] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:14.851487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae63aa8-1688-4a98-9520-296b422c7c8d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:14.875300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:14.891408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9d3da-600a-4022-b6d2-27019606143e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:14.893963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9d3da-600a-4022-b6d2-27019606143e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:14.893963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9d3da-600a-4022-b6d2-27019606143e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:14.893963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9d3da-600a-4022-b6d2-27019606143e None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:14.893963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9d3da-600a-4022-b6d2-27019606143e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:14.893963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9d3da-600a-4022-b6d2-27019606143e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:14.895144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9d3da-600a-4022-b6d2-27019606143e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:14.906097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 502/1005] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 16 05:25:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:14.908294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86febe1-7a44-4a83-8f56-03972437113f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:15.011801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:15.021414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86febe1-7a44-4a83-8f56-03972437113f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:15.056924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89feb4-ebc5-4a6d-8f92-60951e35e73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:15.057525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89feb4-ebc5-4a6d-8f92-60951e35e73f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:15.057798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89feb4-ebc5-4a6d-8f92-60951e35e73f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:15.058092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89feb4-ebc5-4a6d-8f92-60951e35e73f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:15.058774 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89feb4-ebc5-4a6d-8f92-60951e35e73f None admin] RBAC: Adding query filter params (interface=public, service_id=17e04b59859644a58007c5c6da29183d, region_id=RegionOne) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:15.059076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89feb4-ebc5-4a6d-8f92-60951e35e73f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:15.060520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89feb4-ebc5-4a6d-8f92-60951e35e73f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:15.066115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 504/1006] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 16 05:25:14 2022] GET /identity/v3/endpoints?service_id=17e04b59859644a58007c5c6da29183d&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:15.071624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9d3da-600a-4022-b6d2-27019606143e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:15.181742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9d3da-600a-4022-b6d2-27019606143e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:15.217341 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f8923-2ec7-442d-8b5f-95d6c6133226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:15.217979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f8923-2ec7-442d-8b5f-95d6c6133226 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:15.218257 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f8923-2ec7-442d-8b5f-95d6c6133226 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:15.218528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f8923-2ec7-442d-8b5f-95d6c6133226 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:15.219119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f8923-2ec7-442d-8b5f-95d6c6133226 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:15.220467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f8923-2ec7-442d-8b5f-95d6c6133226 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:15.226886 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 503/1007] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 16 05:25:15 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:25:16.180187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d8d4-3e01-4ce4-9eeb-bb46e57fa6b2 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:16.180678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d8d4-3e01-4ce4-9eeb-bb46e57fa6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:16.180678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865d8d4-3e01-4ce4-9eeb-bb46e57fa6b2 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:16.182151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 505/1008] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:16.187337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632be57-7487-414e-9618-9eda8b335270 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:16.187751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632be57-7487-414e-9618-9eda8b335270 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:16.187954 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632be57-7487-414e-9618-9eda8b335270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:16.206396 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-9632be57-7487-414e-9618-9eda8b335270 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:16.207571 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9632be57-7487-414e-9618-9eda8b335270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:16.249753 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:16.272859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 504/1009] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:16 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:16.548884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865d8d4-3e01-4ce4-9eeb-bb46e57fa6b2 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:16.652342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865d8d4-3e01-4ce4-9eeb-bb46e57fa6b2 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:16.733168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a0bd0-0394-40aa-98ec-fdbbb2e2fd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:16.734291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a0bd0-0394-40aa-98ec-fdbbb2e2fd7a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:16.734291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a0bd0-0394-40aa-98ec-fdbbb2e2fd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:16.734291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a0bd0-0394-40aa-98ec-fdbbb2e2fd7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:16.737943 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a0bd0-0394-40aa-98ec-fdbbb2e2fd7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:16.738952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a0bd0-0394-40aa-98ec-fdbbb2e2fd7a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:16.741965 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 506/1010] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:16.746618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632be57-7487-414e-9618-9eda8b335270 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:16.861898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632be57-7487-414e-9618-9eda8b335270 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:16.899635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b2f1c-53a5-4e60-8ee8-fd59524c9fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:16.900641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b2f1c-53a5-4e60-8ee8-fd59524c9fbc None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:16.901117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b2f1c-53a5-4e60-8ee8-fd59524c9fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:16.901581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b2f1c-53a5-4e60-8ee8-fd59524c9fbc None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:16.902518 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b2f1c-53a5-4e60-8ee8-fd59524c9fbc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:16.905262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b2f1c-53a5-4e60-8ee8-fd59524c9fbc None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:16.964100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875f11d-dc86-420d-8d6b-02e71c8c39d4 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:16.964383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875f11d-dc86-420d-8d6b-02e71c8c39d4 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:16.964656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875f11d-dc86-420d-8d6b-02e71c8c39d4 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:16.966049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 507/1011] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:16.971209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29ab0-2317-43c8-9130-c60f9b72e831 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:16.971485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29ab0-2317-43c8-9130-c60f9b72e831 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:16.971752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29ab0-2317-43c8-9130-c60f9b72e831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:16.990124 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 505/1012] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:25:16 2022] POST /identity/v3/projects => generated 311 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:25:16.994609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-fbd29ab0-2317-43c8-9130-c60f9b72e831 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:16.995899 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-fbd29ab0-2317-43c8-9130-c60f9b72e831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:17.017816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:17.076450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 508/1013] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:16 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:17.345761 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b2f1c-53a5-4e60-8ee8-fd59524c9fbc None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:17.377541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:17.413728 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53a700-8d06-49f3-9c21-7581aacca6f0 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:17.414283 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53a700-8d06-49f3-9c21-7581aacca6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:17.414283 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53a700-8d06-49f3-9c21-7581aacca6f0 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:17.415743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 509/1014] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:17.421282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9866c5-7cbf-46d1-9105-345649be9bc7 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:17.421663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9866c5-7cbf-46d1-9105-345649be9bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:17.421874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9866c5-7cbf-46d1-9105-345649be9bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:17.439550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-8f9866c5-7cbf-46d1-9105-345649be9bc7 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:17.440718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8f9866c5-7cbf-46d1-9105-345649be9bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:17.452501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b2f1c-53a5-4e60-8ee8-fd59524c9fbc None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:17.490548 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaacfde-7cac-4b6a-bf54-11a3c23ffc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:17.491146 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaacfde-7cac-4b6a-bf54-11a3c23ffc3d None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:17.491416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaacfde-7cac-4b6a-bf54-11a3c23ffc3d None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:17.491679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaacfde-7cac-4b6a-bf54-11a3c23ffc3d None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:17.492332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaacfde-7cac-4b6a-bf54-11a3c23ffc3d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:17.493677 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaacfde-7cac-4b6a-bf54-11a3c23ffc3d None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:17.513002 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 510/1015] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:17 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:17.578453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 506/1016] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:25:17 2022] POST /identity/v3/services => generated 242 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:25:17.784413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9866c5-7cbf-46d1-9105-345649be9bc7 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:17.869590 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:17.886111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9866c5-7cbf-46d1-9105-345649be9bc7 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:17.930169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc9b81-0286-4248-a3e9-fa6486bc63e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:17.931001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc9b81-0286-4248-a3e9-fa6486bc63e4 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:17.931280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc9b81-0286-4248-a3e9-fa6486bc63e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:17.931548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc9b81-0286-4248-a3e9-fa6486bc63e4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:17.932132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc9b81-0286-4248-a3e9-fa6486bc63e4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:17.933500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc9b81-0286-4248-a3e9-fa6486bc63e4 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-05dc9b81-0286-4248-a3e9-fa6486bc63e4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:17.943375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:17.944406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:17.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 511/1017] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:25:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:17.952286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaacfde-7cac-4b6a-bf54-11a3c23ffc3d None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:18.044477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:18.057999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaacfde-7cac-4b6a-bf54-11a3c23ffc3d None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:18.092685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e140c2-4505-443b-8e63-ddac0873904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:18.093274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e140c2-4505-443b-8e63-ddac0873904a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:18.093548 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e140c2-4505-443b-8e63-ddac0873904a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:18.093816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e140c2-4505-443b-8e63-ddac0873904a None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:18.094463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e140c2-4505-443b-8e63-ddac0873904a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:18.094760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e140c2-4505-443b-8e63-ddac0873904a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:18.096115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e140c2-4505-443b-8e63-ddac0873904a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:18.101004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 507/1018] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 16 05:25:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:18.106529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc9b81-0286-4248-a3e9-fa6486bc63e4 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:18.208516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc9b81-0286-4248-a3e9-fa6486bc63e4 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:18.241660 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ac4b8-df10-46cc-b1c4-1bd49cdc5b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:18.242294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ac4b8-df10-46cc-b1c4-1bd49cdc5b7a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:18.242571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ac4b8-df10-46cc-b1c4-1bd49cdc5b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:18.242840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ac4b8-df10-46cc-b1c4-1bd49cdc5b7a None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:18.243489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ac4b8-df10-46cc-b1c4-1bd49cdc5b7a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:18.243767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ac4b8-df10-46cc-b1c4-1bd49cdc5b7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:18.245125 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ac4b8-df10-46cc-b1c4-1bd49cdc5b7a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:18.250146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 512/1019] 10.222.0.49 () {56 vars in 1112 bytes} [Fri Sep 16 05:25:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:18.255999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e140c2-4505-443b-8e63-ddac0873904a None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:18.358917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e140c2-4505-443b-8e63-ddac0873904a None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:18.392353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de002322-299c-4d3c-88c3-0778c9ac468e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:18.392960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de002322-299c-4d3c-88c3-0778c9ac468e None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:18.393227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de002322-299c-4d3c-88c3-0778c9ac468e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:18.393487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de002322-299c-4d3c-88c3-0778c9ac468e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:18.394181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de002322-299c-4d3c-88c3-0778c9ac468e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:18.395529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de002322-299c-4d3c-88c3-0778c9ac468e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:18.477997 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 508/1020] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:25:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:25:19.211088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950f220-e4b5-458d-8747-e9e2d93c37ea None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:19.211618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950f220-e4b5-458d-8747-e9e2d93c37ea None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:19.211618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950f220-e4b5-458d-8747-e9e2d93c37ea None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:19.213078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 513/1021] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:19.218987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466892e-8b6f-4e5e-bcfe-a6dd25f87389 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:19.219350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466892e-8b6f-4e5e-bcfe-a6dd25f87389 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:19.219708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466892e-8b6f-4e5e-bcfe-a6dd25f87389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:19.237920 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-5466892e-8b6f-4e5e-bcfe-a6dd25f87389 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:19.239110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5466892e-8b6f-4e5e-bcfe-a6dd25f87389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:19.242814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:19.317852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 509/1022] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:19.590017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950f220-e4b5-458d-8747-e9e2d93c37ea None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:19.601611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:19.696080 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950f220-e4b5-458d-8747-e9e2d93c37ea None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:19.739406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8a2b80-34e9-472f-a3d9-b6528b24d7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:19.740457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a2b80-34e9-472f-a3d9-b6528b24d7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:19.740936 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a2b80-34e9-472f-a3d9-b6528b24d7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:19.741404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8a2b80-34e9-472f-a3d9-b6528b24d7a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:19.742743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a2b80-34e9-472f-a3d9-b6528b24d7a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:19.744367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8a2b80-34e9-472f-a3d9-b6528b24d7a6 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:19.746546 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 514/1023] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:19.752098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466892e-8b6f-4e5e-bcfe-a6dd25f87389 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:19.815924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785a7f7-0a0b-4a4b-b039-3b19fbd88fa7 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:19.816246 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785a7f7-0a0b-4a4b-b039-3b19fbd88fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:19.816519 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785a7f7-0a0b-4a4b-b039-3b19fbd88fa7 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:19.818126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 515/1024] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:25:19.822736 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9959e-b2d9-4d9e-b8a7-b26933bd8a6d None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:19.823018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9959e-b2d9-4d9e-b8a7-b26933bd8a6d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:19.823289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9959e-b2d9-4d9e-b8a7-b26933bd8a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:19.843680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-f1c9959e-b2d9-4d9e-b8a7-b26933bd8a6d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:19.844900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f1c9959e-b2d9-4d9e-b8a7-b26933bd8a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:19.860226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466892e-8b6f-4e5e-bcfe-a6dd25f87389 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:19.897649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8f233-9619-476e-9d59-6abc3a9b8501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:19.898557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8f233-9619-476e-9d59-6abc3a9b8501 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:19.899012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8f233-9619-476e-9d59-6abc3a9b8501 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:19.899463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8f233-9619-476e-9d59-6abc3a9b8501 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:19.900130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8f233-9619-476e-9d59-6abc3a9b8501 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:19.901547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8f233-9619-476e-9d59-6abc3a9b8501 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:19.914698 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 516/1025] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:19.915883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:20.015529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 510/1026] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:25:19 2022] POST /identity/v3/users => generated 285 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:25:20.191262 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9959e-b2d9-4d9e-b8a7-b26933bd8a6d None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:20.199514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:20.297709 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9959e-b2d9-4d9e-b8a7-b26933bd8a6d None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:20.332957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd20bf-ad0d-4584-9bd1-20e389300b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:20.333585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd20bf-ad0d-4584-9bd1-20e389300b92 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:20.333891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd20bf-ad0d-4584-9bd1-20e389300b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:20.334173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd20bf-ad0d-4584-9bd1-20e389300b92 None admin] PATH_INFO: `/v3/services/image` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:20.334737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd20bf-ad0d-4584-9bd1-20e389300b92 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:20.336084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd20bf-ad0d-4584-9bd1-20e389300b92 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-58dd20bf-ad0d-4584-9bd1-20e389300b92 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:20.345961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:20.346969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:20.347820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 517/1027] 10.222.0.49 () {56 vars in 1105 bytes} [Fri Sep 16 05:25:20 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:20.353222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8f233-9619-476e-9d59-6abc3a9b8501 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:20.457777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8f233-9619-476e-9d59-6abc3a9b8501 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:20.491914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2fdf5-d40b-4741-9698-c2181dec56d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:20.492518 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2fdf5-d40b-4741-9698-c2181dec56d8 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:20.492794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2fdf5-d40b-4741-9698-c2181dec56d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:20.493061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2fdf5-d40b-4741-9698-c2181dec56d8 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:20.493684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2fdf5-d40b-4741-9698-c2181dec56d8 None admin] RBAC: Adding query filter params (name=image) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:20.494014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2fdf5-d40b-4741-9698-c2181dec56d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:20.495365 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2fdf5-d40b-4741-9698-c2181dec56d8 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:20.500216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 511/1028] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:20.506185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd20bf-ad0d-4584-9bd1-20e389300b92 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:20.576874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:20.607601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd20bf-ad0d-4584-9bd1-20e389300b92 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:20.645987 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb0d8a-dade-4357-8748-cd6250a991f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:20.646611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb0d8a-dade-4357-8748-cd6250a991f3 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:20.646884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb0d8a-dade-4357-8748-cd6250a991f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:20.647148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb0d8a-dade-4357-8748-cd6250a991f3 None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:20.647848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb0d8a-dade-4357-8748-cd6250a991f3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:20.648149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb0d8a-dade-4357-8748-cd6250a991f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:20.649522 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb0d8a-dade-4357-8748-cd6250a991f3 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:20.655733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 518/1029] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:20.661532 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2fdf5-d40b-4741-9698-c2181dec56d8 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:20.719751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:20.737419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93dbad-cc58-41dc-8702-4f89a7a1038b None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:20.737735 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93dbad-cc58-41dc-8702-4f89a7a1038b None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:20.737985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93dbad-cc58-41dc-8702-4f89a7a1038b None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:20.739646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 519/1030] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:25:20.745716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a252f2d-6e55-4e4c-b114-f4d1cf02a347 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:20.746276 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a252f2d-6e55-4e4c-b114-f4d1cf02a347 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:20.746758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a252f2d-6e55-4e4c-b114-f4d1cf02a347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:20.765538 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-1a252f2d-6e55-4e4c-b114-f4d1cf02a347 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:20.766984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-1a252f2d-6e55-4e4c-b114-f4d1cf02a347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:20.774712 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2fdf5-d40b-4741-9698-c2181dec56d8 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:20.816618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69be2f4-7071-48aa-9523-ed7c055dfbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:20.817218 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69be2f4-7071-48aa-9523-ed7c055dfbdf None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:20.817494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69be2f4-7071-48aa-9523-ed7c055dfbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:20.817758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69be2f4-7071-48aa-9523-ed7c055dfbdf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:20.818506 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69be2f4-7071-48aa-9523-ed7c055dfbdf None admin] RBAC: Adding query filter params (interface=public, service_id=89d2a9ba55b74b27b29858c97dbec0bb, region_id=RegionOne) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:20.818803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69be2f4-7071-48aa-9523-ed7c055dfbdf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:20.820144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69be2f4-7071-48aa-9523-ed7c055dfbdf None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:20.830247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 512/1031] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 16 05:25:20 2022] GET /identity/v3/endpoints?service_id=89d2a9ba55b74b27b29858c97dbec0bb&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:20.832287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69be2f4-7071-48aa-9523-ed7c055dfbdf None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:20.853862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 520/1032] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:20.933561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69be2f4-7071-48aa-9523-ed7c055dfbdf None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:20.953855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:20.968044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82573d0c-f548-41af-949d-e45ffc2e8b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:20.968637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82573d0c-f548-41af-949d-e45ffc2e8b4f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:20.972875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82573d0c-f548-41af-949d-e45ffc2e8b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:20.972875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82573d0c-f548-41af-949d-e45ffc2e8b4f None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:20.972875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82573d0c-f548-41af-949d-e45ffc2e8b4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:20.972875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82573d0c-f548-41af-949d-e45ffc2e8b4f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:20.978838 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 513/1033] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 16 05:25:20 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:25:21.112622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a252f2d-6e55-4e4c-b114-f4d1cf02a347 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:21.180828 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:21.217956 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a252f2d-6e55-4e4c-b114-f4d1cf02a347 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:21.252505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7e851-b4d7-4f4d-be3f-0b66c765a498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:21.253047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7e851-b4d7-4f4d-be3f-0b66c765a498 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:21.253315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7e851-b4d7-4f4d-be3f-0b66c765a498 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:21.253579 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7e851-b4d7-4f4d-be3f-0b66c765a498 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:21.264176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7e851-b4d7-4f4d-be3f-0b66c765a498 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-06f7e851-b4d7-4f4d-be3f-0b66c765a498 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:25:21.269201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:21.270459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:21.271737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:21.271737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:21.271737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:21.271737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:21.271737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:25:21.271737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:25:21.271737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:25:21.271737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:25:21.271737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:25:21.271737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:25:21.271737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 05:25:21.271737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:21.271737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 521/1034] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Sep 16 05:25:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:21.276446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82573d0c-f548-41af-949d-e45ffc2e8b4f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:21.375650 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82573d0c-f548-41af-949d-e45ffc2e8b4f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:21.409834 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6335bc-cbfe-4acd-af68-3eddc5197d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:21.410482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6335bc-cbfe-4acd-af68-3eddc5197d59 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:21.410777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6335bc-cbfe-4acd-af68-3eddc5197d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:21.411040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6335bc-cbfe-4acd-af68-3eddc5197d59 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:21.411694 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6335bc-cbfe-4acd-af68-3eddc5197d59 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:21.411989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6335bc-cbfe-4acd-af68-3eddc5197d59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:21.413335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6335bc-cbfe-4acd-af68-3eddc5197d59 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:21.426108 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 514/1035] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 16 05:25:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:21.427873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7e851-b4d7-4f4d-be3f-0b66c765a498 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:21.529791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7e851-b4d7-4f4d-be3f-0b66c765a498 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:21.553132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:21.564313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82eb1d-2661-4736-9681-67550f61805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:21.569068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82eb1d-2661-4736-9681-67550f61805f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:21.569068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82eb1d-2661-4736-9681-67550f61805f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:21.569068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82eb1d-2661-4736-9681-67550f61805f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:21.574946 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c82eb1d-2661-4736-9681-67550f61805f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:21.576992 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c82eb1d-2661-4736-9681-67550f61805f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-8c82eb1d-2661-4736-9681-67550f61805f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:25:21.591287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:21.597301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:25:21.598281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:25:21.598281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:21.598281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:21.598281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 522/1036] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:21.600340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6335bc-cbfe-4acd-af68-3eddc5197d59 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:21.646935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:21.700319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6335bc-cbfe-4acd-af68-3eddc5197d59 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:21.734294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac767b-ddd5-490a-a176-41d65a793d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:21.734892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac767b-ddd5-490a-a176-41d65a793d01 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:21.735161 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac767b-ddd5-490a-a176-41d65a793d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:21.735421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac767b-ddd5-490a-a176-41d65a793d01 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:21.736085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac767b-ddd5-490a-a176-41d65a793d01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:21.736361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac767b-ddd5-490a-a176-41d65a793d01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:21.737690 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac767b-ddd5-490a-a176-41d65a793d01 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:21.749608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 515/1037] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:21.755013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82eb1d-2661-4736-9681-67550f61805f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:21.854681 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82eb1d-2661-4736-9681-67550f61805f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:21.893102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de86c2-30b6-42e1-a665-3318a38e89ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:21.893725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de86c2-30b6-42e1-a665-3318a38e89ca None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:21.894027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de86c2-30b6-42e1-a665-3318a38e89ca None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:21.894294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de86c2-30b6-42e1-a665-3318a38e89ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:21.901791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de86c2-30b6-42e1-a665-3318a38e89ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:21.903231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de86c2-30b6-42e1-a665-3318a38e89ca None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-e0de86c2-30b6-42e1-a665-3318a38e89ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:25:21.915181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:21.916261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:21.917582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:21.917582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:21.917582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:25:21.917582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:25:21.917582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:25:21.917582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:25:21.917582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:25:21.917582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:25:21.917582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:25:21.917582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:21.921895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 523/1038] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 16 05:25:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:21.923159 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac767b-ddd5-490a-a176-41d65a793d01 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:22.026727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac767b-ddd5-490a-a176-41d65a793d01 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:22.035047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:22.061302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c074432-b60b-467c-a127-30e16eb1ae71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:22.065774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c074432-b60b-467c-a127-30e16eb1ae71 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:22.065774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c074432-b60b-467c-a127-30e16eb1ae71 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:22.065774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c074432-b60b-467c-a127-30e16eb1ae71 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:22.065774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c074432-b60b-467c-a127-30e16eb1ae71 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:22.065774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c074432-b60b-467c-a127-30e16eb1ae71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:22.065774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c074432-b60b-467c-a127-30e16eb1ae71 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:22.087324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 516/1039] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 16 05:25:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:22.088453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de86c2-30b6-42e1-a665-3318a38e89ca None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:22.126045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:22.194115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de86c2-30b6-42e1-a665-3318a38e89ca None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:22.231064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd2fa2-ea46-41b1-8c02-a443e1f14f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:22.231670 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd2fa2-ea46-41b1-8c02-a443e1f14f97 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:22.232493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd2fa2-ea46-41b1-8c02-a443e1f14f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:22.232764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd2fa2-ea46-41b1-8c02-a443e1f14f97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:22.299375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372e99e-c537-42a1-b4f6-981d8b8538da None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:22.299649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372e99e-c537-42a1-b4f6-981d8b8538da None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:22.299920 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372e99e-c537-42a1-b4f6-981d8b8538da None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:22.301254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 517/1040] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:22.302393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd2fa2-ea46-41b1-8c02-a443e1f14f97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:22.303846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd2fa2-ea46-41b1-8c02-a443e1f14f97 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:22.307516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aebee2-33bf-4bc8-8e35-d5b5e1cea70a None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:22.307797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aebee2-33bf-4bc8-8e35-d5b5e1cea70a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:22.308069 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aebee2-33bf-4bc8-8e35-d5b5e1cea70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:22.327638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f4aebee2-33bf-4bc8-8e35-d5b5e1cea70a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:22.328816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f4aebee2-33bf-4bc8-8e35-d5b5e1cea70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:22.337721 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:22.400494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 518/1041] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:22.410345 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 524/1042] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:25:22 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:25:22.411397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aebee2-33bf-4bc8-8e35-d5b5e1cea70a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:22.505528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:22.512317 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aebee2-33bf-4bc8-8e35-d5b5e1cea70a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:22.547260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775eb44-503f-4a47-bd6a-ef1ec2274264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:22.548410 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775eb44-503f-4a47-bd6a-ef1ec2274264 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:22.549039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775eb44-503f-4a47-bd6a-ef1ec2274264 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:22.549880 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775eb44-503f-4a47-bd6a-ef1ec2274264 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:22.560577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775eb44-503f-4a47-bd6a-ef1ec2274264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:22.562028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775eb44-503f-4a47-bd6a-ef1ec2274264 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-e775eb44-503f-4a47-bd6a-ef1ec2274264 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:25:22.576780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:22.577733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:25:22.579379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:25:22.579379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:22.579379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:22.579379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 519/1043] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:22.587003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd2fa2-ea46-41b1-8c02-a443e1f14f97 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:22.680237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775eb44-503f-4a47-bd6a-ef1ec2274264 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:22.687625 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd2fa2-ea46-41b1-8c02-a443e1f14f97 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:22.723300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f9210-a4ab-497e-b1cd-88b4ae02d613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:22.724371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f9210-a4ab-497e-b1cd-88b4ae02d613 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:22.724849 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f9210-a4ab-497e-b1cd-88b4ae02d613 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:22.725482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f9210-a4ab-497e-b1cd-88b4ae02d613 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:22.726157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f9210-a4ab-497e-b1cd-88b4ae02d613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:22.726457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f9210-a4ab-497e-b1cd-88b4ae02d613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:22.727807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f9210-a4ab-497e-b1cd-88b4ae02d613 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:22.730823 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:22.745598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 525/1044] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:22.746631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f9210-a4ab-497e-b1cd-88b4ae02d613 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:22.808456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775eb44-503f-4a47-bd6a-ef1ec2274264 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:22.849917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a5464-b117-4792-ae0a-037a13fb1973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:22.850937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a5464-b117-4792-ae0a-037a13fb1973 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:22.851498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a5464-b117-4792-ae0a-037a13fb1973 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:22.851771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a5464-b117-4792-ae0a-037a13fb1973 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:22.864376 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a5464-b117-4792-ae0a-037a13fb1973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-e89a5464-b117-4792-ae0a-037a13fb1973 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:25:22.869147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:22.870316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:22.871194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:22.871194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:22.871194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:22.871194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:22.871194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:25:22.871194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:25:22.871194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:25:22.871194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:25:22.871194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:25:22.871194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:25:22.871194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:25:22.871194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:22.871887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f9210-a4ab-497e-b1cd-88b4ae02d613 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:22.871972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 520/1045] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:25:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:22.876525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a5464-b117-4792-ae0a-037a13fb1973 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:22.886324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:22.913762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cee71c-697f-46d2-bc0b-30f52b610322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:22.914513 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cee71c-697f-46d2-bc0b-30f52b610322 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:22.914859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cee71c-697f-46d2-bc0b-30f52b610322 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:22.915197 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cee71c-697f-46d2-bc0b-30f52b610322 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:22.926426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cee71c-697f-46d2-bc0b-30f52b610322 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:22.927894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cee71c-697f-46d2-bc0b-30f52b610322 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-e9cee71c-697f-46d2-bc0b-30f52b610322 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:25:22.939514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:22.940406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:22.941326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:25:22.941326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:25:22.941326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:25:22.941326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:25:22.941326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:25:22.941326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:25:22.941326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:22.941326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:22.941326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 526/1046] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:25:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:22.946333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cee71c-697f-46d2-bc0b-30f52b610322 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:22.992515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a5464-b117-4792-ae0a-037a13fb1973 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:23.004592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:23.030856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c41652-b640-4d8a-8407-480038a46862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:23.031738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c41652-b640-4d8a-8407-480038a46862 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:23.032357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c41652-b640-4d8a-8407-480038a46862 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:23.032780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c41652-b640-4d8a-8407-480038a46862 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:23.034115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c41652-b640-4d8a-8407-480038a46862 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:23.034695 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c41652-b640-4d8a-8407-480038a46862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:23.036208 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c41652-b640-4d8a-8407-480038a46862 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:23.049874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 521/1047] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:25:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:23.054257 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c41652-b640-4d8a-8407-480038a46862 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:23.056490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cee71c-697f-46d2-bc0b-30f52b610322 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:23.094361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86218f86-84ca-4376-b878-0976a0592d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:23.094960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86218f86-84ca-4376-b878-0976a0592d17 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:23.095228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86218f86-84ca-4376-b878-0976a0592d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:23.095490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86218f86-84ca-4376-b878-0976a0592d17 None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:23.096166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86218f86-84ca-4376-b878-0976a0592d17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:23.096460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86218f86-84ca-4376-b878-0976a0592d17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:23.097876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86218f86-84ca-4376-b878-0976a0592d17 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:23.116014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 527/1048] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:25:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:23.124918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86218f86-84ca-4376-b878-0976a0592d17 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:23.133657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:23.176517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c41652-b640-4d8a-8407-480038a46862 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:23.211433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a946bf6-7023-41f9-a0ca-ffcb20a5386e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:23.211974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a946bf6-7023-41f9-a0ca-ffcb20a5386e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:23.212251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a946bf6-7023-41f9-a0ca-ffcb20a5386e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:23.212516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a946bf6-7023-41f9-a0ca-ffcb20a5386e None admin] PATH_INFO: `/v3/users/dff0fc41ad6c4a288ca455180bd111fd` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:23.226050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a946bf6-7023-41f9-a0ca-ffcb20a5386e None admin] RBAC: Authorizing `identity:get_user(user_id=dff0fc41ad6c4a288ca455180bd111fd)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:23.227644 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a946bf6-7023-41f9-a0ca-ffcb20a5386e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:23.229270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 522/1049] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:25:23 2022] GET /identity/v3/users/dff0fc41ad6c4a288ca455180bd111fd => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:23.230452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13350bd5-899e-43f1-b9e9-59aeb8a9b3ef None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:23.230746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13350bd5-899e-43f1-b9e9-59aeb8a9b3ef None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:23.231000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13350bd5-899e-43f1-b9e9-59aeb8a9b3ef None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:23.232592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 523/1050] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:23.232773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86218f86-84ca-4376-b878-0976a0592d17 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:23.235251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13350bd5-899e-43f1-b9e9-59aeb8a9b3ef None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:23.250189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:23.268155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47688d-3421-48d7-93b2-f7e8ac997519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:23.268780 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47688d-3421-48d7-93b2-f7e8ac997519 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:23.269055 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47688d-3421-48d7-93b2-f7e8ac997519 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:23.269319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47688d-3421-48d7-93b2-f7e8ac997519 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:23.270059 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47688d-3421-48d7-93b2-f7e8ac997519 None admin] RBAC: Adding query filter params (role.id=6851de0f376f41a8a000506d1df67181, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=7db75c20e9744feb8b5dc78c554f4977) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:23.270362 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47688d-3421-48d7-93b2-f7e8ac997519 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:23.271768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47688d-3421-48d7-93b2-f7e8ac997519 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:23.277164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 528/1051] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:25:23 2022] GET /identity/v3/role_assignments?user.id=7db75c20e9744feb8b5dc78c554f4977&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=6851de0f376f41a8a000506d1df67181 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:23.278132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1357f-93c1-4ada-bacf-4573893e4c7d None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:23.278409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1357f-93c1-4ada-bacf-4573893e4c7d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:23.278675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1357f-93c1-4ada-bacf-4573893e4c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:23.297869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-41b1357f-93c1-4ada-bacf-4573893e4c7d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:23.299109 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-41b1357f-93c1-4ada-bacf-4573893e4c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:23.351795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13350bd5-899e-43f1-b9e9-59aeb8a9b3ef None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:23.374812 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 529/1052] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:23.383121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:23.392615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fd619-9fd3-4e2b-a1bb-5a89d2814701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:23.393240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fd619-9fd3-4e2b-a1bb-5a89d2814701 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:23.396935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fd619-9fd3-4e2b-a1bb-5a89d2814701 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:23.396935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fd619-9fd3-4e2b-a1bb-5a89d2814701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:23.470832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fd619-9fd3-4e2b-a1bb-5a89d2814701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:23.472189 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fd619-9fd3-4e2b-a1bb-5a89d2814701 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:23.581089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 524/1053] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:25:23 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 16 05:25:23.582377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1357f-93c1-4ada-bacf-4573893e4c7d None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:23.651884 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fd619-9fd3-4e2b-a1bb-5a89d2814701 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:23.680705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:23.683834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1357f-93c1-4ada-bacf-4573893e4c7d None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:23.694803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:23.719055 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f411300-00bd-408f-9847-1896b9d192d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:23.719690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f411300-00bd-408f-9847-1896b9d192d1 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:23.719966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f411300-00bd-408f-9847-1896b9d192d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:23.720237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f411300-00bd-408f-9847-1896b9d192d1 None admin] PATH_INFO: `/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:23.721413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f411300-00bd-408f-9847-1896b9d192d1 None admin] RBAC: Authorizing `identity:get_project(project_id=3f951307f9bc405cbc7fca51f6b4fae1)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:23.722816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f411300-00bd-408f-9847-1896b9d192d1 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:23.729451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 530/1054] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:25:23 2022] GET /identity/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:23.730523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f411300-00bd-408f-9847-1896b9d192d1 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:23.769885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fd619-9fd3-4e2b-a1bb-5a89d2814701 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:23.807937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda81590-79e9-4ed5-bd59-046fa38a4527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:23.808564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda81590-79e9-4ed5-bd59-046fa38a4527 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:23.808841 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda81590-79e9-4ed5-bd59-046fa38a4527 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:23.809109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda81590-79e9-4ed5-bd59-046fa38a4527 None admin] PATH_INFO: `/v3/services/image` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:23.809684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda81590-79e9-4ed5-bd59-046fa38a4527 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:23.811614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda81590-79e9-4ed5-bd59-046fa38a4527 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-cda81590-79e9-4ed5-bd59-046fa38a4527 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:23.820927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:23.821957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:23.822811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 525/1055] 10.222.0.49 () {56 vars in 1105 bytes} [Fri Sep 16 05:25:23 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:23.828384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda81590-79e9-4ed5-bd59-046fa38a4527 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:23.841000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f411300-00bd-408f-9847-1896b9d192d1 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:23.875406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab78f41-70ed-47f3-af9a-57386607c532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:23.876012 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab78f41-70ed-47f3-af9a-57386607c532 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:23.876271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab78f41-70ed-47f3-af9a-57386607c532 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:23.876532 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab78f41-70ed-47f3-af9a-57386607c532 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:23.877236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab78f41-70ed-47f3-af9a-57386607c532 None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=3f951307f9bc405cbc7fca51f6b4fae1, user.id=dff0fc41ad6c4a288ca455180bd111fd) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:23.877533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab78f41-70ed-47f3-af9a-57386607c532 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:23.879954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab78f41-70ed-47f3-af9a-57386607c532 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:23.885395 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 531/1056] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:25:23 2022] GET /identity/v3/role_assignments?user.id=dff0fc41ad6c4a288ca455180bd111fd&scope.project.id=3f951307f9bc405cbc7fca51f6b4fae1&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:23.938556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:23.949097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda81590-79e9-4ed5-bd59-046fa38a4527 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:23.987179 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4540e-0940-4dc8-be95-1179462e8e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:23.987705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4540e-0940-4dc8-be95-1179462e8e72 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:23.987972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4540e-0940-4dc8-be95-1179462e8e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:23.988235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4540e-0940-4dc8-be95-1179462e8e72 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:23.988865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4540e-0940-4dc8-be95-1179462e8e72 None admin] RBAC: Adding query filter params (name=image) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:23.989225 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4540e-0940-4dc8-be95-1179462e8e72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:23.990676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd4540e-0940-4dc8-be95-1179462e8e72 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:23.996461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 526/1057] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:24.002847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab78f41-70ed-47f3-af9a-57386607c532 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:24.051564 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:24.110551 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab78f41-70ed-47f3-af9a-57386607c532 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:24.144378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67242abb-d39d-4877-b75d-e7d2cd409f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:24.145042 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67242abb-d39d-4877-b75d-e7d2cd409f1a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:24.145312 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67242abb-d39d-4877-b75d-e7d2cd409f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:24.145576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67242abb-d39d-4877-b75d-e7d2cd409f1a None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:24.146281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67242abb-d39d-4877-b75d-e7d2cd409f1a None admin] RBAC: Adding query filter params (type=image) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:24.146575 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67242abb-d39d-4877-b75d-e7d2cd409f1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:24.147963 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67242abb-d39d-4877-b75d-e7d2cd409f1a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:24.159056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 532/1058] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:24.160824 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4540e-0940-4dc8-be95-1179462e8e72 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:24.261864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4540e-0940-4dc8-be95-1179462e8e72 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:24.295350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756d087-60b2-46d0-a9e0-1779dd615d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:24.295942 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756d087-60b2-46d0-a9e0-1779dd615d9c None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:24.296210 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756d087-60b2-46d0-a9e0-1779dd615d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:24.296470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756d087-60b2-46d0-a9e0-1779dd615d9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:24.297123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756d087-60b2-46d0-a9e0-1779dd615d9c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:24.298497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756d087-60b2-46d0-a9e0-1779dd615d9c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:24.406546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 527/1059] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:25:24 2022] POST /identity/v3/endpoints => generated 327 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:25:25.475089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc51562-c82d-4a1c-a641-4e873ae8e4aa None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:25.475609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc51562-c82d-4a1c-a641-4e873ae8e4aa None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:25.475609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc51562-c82d-4a1c-a641-4e873ae8e4aa None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:25.477001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 533/1060] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:25.482631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0489d-774b-48db-b302-0a5d3ed970a1 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:25.483089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0489d-774b-48db-b302-0a5d3ed970a1 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:25.483349 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0489d-774b-48db-b302-0a5d3ed970a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:25.501379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-69c0489d-774b-48db-b302-0a5d3ed970a1 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:25.502562 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-69c0489d-774b-48db-b302-0a5d3ed970a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:25.506138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:25.579763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 528/1061] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:25.846499 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc51562-c82d-4a1c-a641-4e873ae8e4aa None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:25.955033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc51562-c82d-4a1c-a641-4e873ae8e4aa None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:25.963291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:25.993067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb655c8-04c9-4b0a-9264-6a664f9d4f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:25.994326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb655c8-04c9-4b0a-9264-6a664f9d4f89 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:25.994973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb655c8-04c9-4b0a-9264-6a664f9d4f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:25.995753 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb655c8-04c9-4b0a-9264-6a664f9d4f89 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:26.007481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb655c8-04c9-4b0a-9264-6a664f9d4f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-bdb655c8-04c9-4b0a-9264-6a664f9d4f89 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:25:26.012374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:26.013937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:26.018754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:26.018754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:26.018754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:26.018754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:26.018754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:25:26.018754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:25:26.018754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:25:26.018754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:25:26.018754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:25:26.018754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:25:26.018754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 05:25:26.018754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:26.018754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 534/1062] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Sep 16 05:25:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:26.020802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0489d-774b-48db-b302-0a5d3ed970a1 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:26.092432 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f944377-721a-4ca9-b1b2-7a77596bb207 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:26.092729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f944377-721a-4ca9-b1b2-7a77596bb207 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:26.092957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f944377-721a-4ca9-b1b2-7a77596bb207 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:26.094326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 535/1063] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:26.099393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d907ff-a235-4794-abb9-57bb6baf3db1 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:26.099614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d907ff-a235-4794-abb9-57bb6baf3db1 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:26.099885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d907ff-a235-4794-abb9-57bb6baf3db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:26.118138 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-04d907ff-a235-4794-abb9-57bb6baf3db1 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:26.119329 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-04d907ff-a235-4794-abb9-57bb6baf3db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:26.124038 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0489d-774b-48db-b302-0a5d3ed970a1 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:26.162822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312958e0-e108-4ea1-ae5a-ab8369b1984c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:26.163430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312958e0-e108-4ea1-ae5a-ab8369b1984c None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:26.163706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312958e0-e108-4ea1-ae5a-ab8369b1984c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:26.163975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312958e0-e108-4ea1-ae5a-ab8369b1984c None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:26.164643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312958e0-e108-4ea1-ae5a-ab8369b1984c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:26.164943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312958e0-e108-4ea1-ae5a-ab8369b1984c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:26.166335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312958e0-e108-4ea1-ae5a-ab8369b1984c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:26.169195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:26.181187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 529/1064] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 16 05:25:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:26.183042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312958e0-e108-4ea1-ae5a-ab8369b1984c None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:26.199326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 536/1065] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:26.292881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312958e0-e108-4ea1-ae5a-ab8369b1984c None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:26.326453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4badae-e2f2-475d-b705-c5f80d6a3139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:26.327123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4badae-e2f2-475d-b705-c5f80d6a3139 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:26.327393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4badae-e2f2-475d-b705-c5f80d6a3139 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:26.327653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4badae-e2f2-475d-b705-c5f80d6a3139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:26.338137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4badae-e2f2-475d-b705-c5f80d6a3139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:26.339550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4badae-e2f2-475d-b705-c5f80d6a3139 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-6b4badae-e2f2-475d-b705-c5f80d6a3139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:25:26.353458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:26.359541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:25:26.360701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:25:26.360701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:26.360701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:26.360701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 530/1066] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:26.362316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d907ff-a235-4794-abb9-57bb6baf3db1 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:26.454127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:26.460311 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d907ff-a235-4794-abb9-57bb6baf3db1 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:26.474537 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4badae-e2f2-475d-b705-c5f80d6a3139 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:26.507099 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f1673-31f4-408a-8285-3ad45a20b720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:26.507737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f1673-31f4-408a-8285-3ad45a20b720 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:26.508006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f1673-31f4-408a-8285-3ad45a20b720 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:26.508279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f1673-31f4-408a-8285-3ad45a20b720 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:26.508940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f1673-31f4-408a-8285-3ad45a20b720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:26.509242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f1673-31f4-408a-8285-3ad45a20b720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:26.510668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f1673-31f4-408a-8285-3ad45a20b720 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:26.523224 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 537/1067] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:26.528747 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f1673-31f4-408a-8285-3ad45a20b720 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:26.543935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:26.598579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4badae-e2f2-475d-b705-c5f80d6a3139 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:26.632231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f1673-31f4-408a-8285-3ad45a20b720 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:26.633741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7633761c-974e-421f-bbb2-aa0281a98989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:26.634467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633761c-974e-421f-bbb2-aa0281a98989 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:26.634736 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633761c-974e-421f-bbb2-aa0281a98989 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:26.634997 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633761c-974e-421f-bbb2-aa0281a98989 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:26.645632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7633761c-974e-421f-bbb2-aa0281a98989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-7633761c-974e-421f-bbb2-aa0281a98989 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:25:26.650377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:26.652305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 531/1068] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:25:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:26.653387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795891cb-d012-4a6d-a800-7e717e9dceca None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:26.654556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795891cb-d012-4a6d-a800-7e717e9dceca None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:26.654556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795891cb-d012-4a6d-a800-7e717e9dceca None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:26.654556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 532/1069] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:26.657502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795891cb-d012-4a6d-a800-7e717e9dceca None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:26.668724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca57f7c4-b5d5-48ce-9606-cd040c9203ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:26.669893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57f7c4-b5d5-48ce-9606-cd040c9203ec None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:26.670174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57f7c4-b5d5-48ce-9606-cd040c9203ec None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:26.670444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca57f7c4-b5d5-48ce-9606-cd040c9203ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:26.677845 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57f7c4-b5d5-48ce-9606-cd040c9203ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:26.679270 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca57f7c4-b5d5-48ce-9606-cd040c9203ec None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-ca57f7c4-b5d5-48ce-9606-cd040c9203ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:25:26.690958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:26.691953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:26.692920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:26.692920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:26.692920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:25:26.692920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:25:26.692920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:25:26.692920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:25:26.692920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:25:26.692920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:25:26.692920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:25:26.692920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:26.692920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 538/1070] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 16 05:25:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:26.693530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d0eac-1f65-47e1-9999-c357ae98c073 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:26.693530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d0eac-1f65-47e1-9999-c357ae98c073 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:26.693653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d0eac-1f65-47e1-9999-c357ae98c073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:26.710436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-2f0d0eac-1f65-47e1-9999-c357ae98c073 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:26.711596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2f0d0eac-1f65-47e1-9999-c357ae98c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:26.721995 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:26.763826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795891cb-d012-4a6d-a800-7e717e9dceca None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:26.775873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:26.781545 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 539/1071] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:26.782451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d0eac-1f65-47e1-9999-c357ae98c073 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:26.804205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a95060-2f10-40e7-ba1c-5572173edcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:26.804803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a95060-2f10-40e7-ba1c-5572173edcaa None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:26.811929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a95060-2f10-40e7-ba1c-5572173edcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:26.811929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a95060-2f10-40e7-ba1c-5572173edcaa None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:26.811929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a95060-2f10-40e7-ba1c-5572173edcaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:26.811929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a95060-2f10-40e7-ba1c-5572173edcaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:26.811929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a95060-2f10-40e7-ba1c-5572173edcaa None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:26.823283 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 533/1072] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:25:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:26.824520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a95060-2f10-40e7-ba1c-5572173edcaa None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:26.895669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d0eac-1f65-47e1-9999-c357ae98c073 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:26.932798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b2991-c663-4a5e-ba95-7de7e68b0fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:26.933402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b2991-c663-4a5e-ba95-7de7e68b0fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:26.933679 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b2991-c663-4a5e-ba95-7de7e68b0fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:26.934156 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b2991-c663-4a5e-ba95-7de7e68b0fa3 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:26.934693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b2991-c663-4a5e-ba95-7de7e68b0fa3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:26.935535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b2991-c663-4a5e-ba95-7de7e68b0fa3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:26.936938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b2991-c663-4a5e-ba95-7de7e68b0fa3 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:26.941857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a95060-2f10-40e7-ba1c-5572173edcaa None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:26.955819 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 540/1073] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 16 05:25:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:26.961451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b2991-c663-4a5e-ba95-7de7e68b0fa3 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:26.965096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:26.982207 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ce2f5-5a6d-4b97-aebb-8028b749df08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:26.982884 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ce2f5-5a6d-4b97-aebb-8028b749df08 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:26.983228 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ce2f5-5a6d-4b97-aebb-8028b749df08 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:26.983490 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ce2f5-5a6d-4b97-aebb-8028b749df08 None admin] PATH_INFO: `/v3/users/dff0fc41ad6c4a288ca455180bd111fd` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:26.984726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ce2f5-5a6d-4b97-aebb-8028b749df08 None admin] RBAC: Authorizing `identity:get_user(user_id=dff0fc41ad6c4a288ca455180bd111fd)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:26.986305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ce2f5-5a6d-4b97-aebb-8028b749df08 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:26.987832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 534/1074] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:25:26 2022] GET /identity/v3/users/dff0fc41ad6c4a288ca455180bd111fd => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:26.993314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ce2f5-5a6d-4b97-aebb-8028b749df08 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:27.040210 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:27.068059 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b2991-c663-4a5e-ba95-7de7e68b0fa3 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:27.100528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ce2f5-5a6d-4b97-aebb-8028b749df08 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:27.102167 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27846691-7c78-41f4-9c5d-da042adb5383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:27.102792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27846691-7c78-41f4-9c5d-da042adb5383 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:27.103066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27846691-7c78-41f4-9c5d-da042adb5383 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:27.103334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27846691-7c78-41f4-9c5d-da042adb5383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:27.113161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27846691-7c78-41f4-9c5d-da042adb5383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:27.114641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27846691-7c78-41f4-9c5d-da042adb5383 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-27846691-7c78-41f4-9c5d-da042adb5383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:25:27.129589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:27.130731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:25:27.131630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:25:27.131630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:27.131630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:27.131630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 541/1075] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:27.131630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27846691-7c78-41f4-9c5d-da042adb5383 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:27.140303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5596a-172c-4976-b273-29adb6552fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:27.140935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5596a-172c-4976-b273-29adb6552fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:27.141207 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5596a-172c-4976-b273-29adb6552fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:27.141474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5596a-172c-4976-b273-29adb6552fc8 None admin] PATH_INFO: `/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:27.142867 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5596a-172c-4976-b273-29adb6552fc8 None admin] RBAC: Authorizing `identity:get_project(project_id=3f951307f9bc405cbc7fca51f6b4fae1)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:27.144284 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5596a-172c-4976-b273-29adb6552fc8 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:27.151166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 535/1076] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:25:26 2022] GET /identity/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:27.151166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5596a-172c-4976-b273-29adb6552fc8 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:27.191247 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:27.234750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27846691-7c78-41f4-9c5d-da042adb5383 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:27.249805 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5596a-172c-4976-b273-29adb6552fc8 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:27.260746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:27.268933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aeb027-1aba-4872-9a75-ebb255226943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:27.269528 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aeb027-1aba-4872-9a75-ebb255226943 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:27.269803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aeb027-1aba-4872-9a75-ebb255226943 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:27.270096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aeb027-1aba-4872-9a75-ebb255226943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:27.285302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6d853-59ae-4999-b0ac-1eef087a6e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:27.286026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6d853-59ae-4999-b0ac-1eef087a6e74 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:27.286370 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6d853-59ae-4999-b0ac-1eef087a6e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:27.286639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6d853-59ae-4999-b0ac-1eef087a6e74 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:27.287273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6d853-59ae-4999-b0ac-1eef087a6e74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:27.287573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6d853-59ae-4999-b0ac-1eef087a6e74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:27.288924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6d853-59ae-4999-b0ac-1eef087a6e74 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:27.300879 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 536/1077] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:27.301627 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6d853-59ae-4999-b0ac-1eef087a6e74 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:27.338764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aeb027-1aba-4872-9a75-ebb255226943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:27.340200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aeb027-1aba-4872-9a75-ebb255226943 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:27.395106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:27.397326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6d853-59ae-4999-b0ac-1eef087a6e74 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:27.431030 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f8afd-fd98-4c06-af17-8656d113958f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:27.431593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f8afd-fd98-4c06-af17-8656d113958f None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:27.431869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f8afd-fd98-4c06-af17-8656d113958f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:27.432129 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f8afd-fd98-4c06-af17-8656d113958f None admin] PATH_INFO: `/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1/users/dff0fc41ad6c4a288ca455180bd111fd/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:27.434013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f8afd-fd98-4c06-af17-8656d113958f None admin] RBAC: Authorizing `identity:create_grant(project_id=3f951307f9bc405cbc7fca51f6b4fae1, user_id=dff0fc41ad6c4a288ca455180bd111fd, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:27.436080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f8afd-fd98-4c06-af17-8656d113958f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:27.439807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 542/1078] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:25:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:25:27.440738 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aeb027-1aba-4872-9a75-ebb255226943 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:27.485693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 537/1079] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:25:27 2022] PUT /identity/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1/users/dff0fc41ad6c4a288ca455180bd111fd/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:25:27.486350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f8afd-fd98-4c06-af17-8656d113958f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:27.537810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:27.547286 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aeb027-1aba-4872-9a75-ebb255226943 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:27.587440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c32a8c-2a35-46db-a2cb-fe975c376b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:27.588077 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c32a8c-2a35-46db-a2cb-fe975c376b47 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:27.588348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c32a8c-2a35-46db-a2cb-fe975c376b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:27.588609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c32a8c-2a35-46db-a2cb-fe975c376b47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:27.596598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c32a8c-2a35-46db-a2cb-fe975c376b47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:27.598081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c32a8c-2a35-46db-a2cb-fe975c376b47 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-27c32a8c-2a35-46db-a2cb-fe975c376b47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:25:27.610018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:27.611180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:27.612251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:25:27.612251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:25:27.612251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:25:27.612251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:25:27.612251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:25:27.612251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:25:27.612251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:27.612251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:27.616165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 543/1080] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:25:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:27.616939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c32a8c-2a35-46db-a2cb-fe975c376b47 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:27.622933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f8afd-fd98-4c06-af17-8656d113958f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:27.636387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:27.665844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a47dff-9604-4cd2-a0e8-760e4ee37246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:27.665844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a47dff-9604-4cd2-a0e8-760e4ee37246 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:27.666266 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a47dff-9604-4cd2-a0e8-760e4ee37246 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:27.666416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a47dff-9604-4cd2-a0e8-760e4ee37246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:27.680956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a47dff-9604-4cd2-a0e8-760e4ee37246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:27.682508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a47dff-9604-4cd2-a0e8-760e4ee37246 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-89a47dff-9604-4cd2-a0e8-760e4ee37246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:25:27.698288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:27.699488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:25:27.700710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:25:27.700710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:27.700710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:27.700710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 538/1081] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:27.707190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a47dff-9604-4cd2-a0e8-760e4ee37246 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:27.729723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c32a8c-2a35-46db-a2cb-fe975c376b47 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:27.764164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97da09-427c-403a-89ef-4f91e0b57811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:27.764780 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97da09-427c-403a-89ef-4f91e0b57811 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:27.765059 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97da09-427c-403a-89ef-4f91e0b57811 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:27.765325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97da09-427c-403a-89ef-4f91e0b57811 None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:27.766039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97da09-427c-403a-89ef-4f91e0b57811 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:27.766342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97da09-427c-403a-89ef-4f91e0b57811 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:27.768028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97da09-427c-403a-89ef-4f91e0b57811 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:27.780636 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 544/1082] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:25:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:27.786434 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97da09-427c-403a-89ef-4f91e0b57811 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:27.794008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:27.813744 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a47dff-9604-4cd2-a0e8-760e4ee37246 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:27.854669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135f747-cc5a-42db-b6ed-98ef54b80c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:27.855855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135f747-cc5a-42db-b6ed-98ef54b80c18 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:27.856137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135f747-cc5a-42db-b6ed-98ef54b80c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:27.856408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135f747-cc5a-42db-b6ed-98ef54b80c18 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:27.857584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135f747-cc5a-42db-b6ed-98ef54b80c18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:27.857918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135f747-cc5a-42db-b6ed-98ef54b80c18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:27.859675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135f747-cc5a-42db-b6ed-98ef54b80c18 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:27.865761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:27.873150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 539/1083] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:27.890751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97da09-427c-403a-89ef-4f91e0b57811 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:27.937237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b0487-572d-453b-af08-a5ac9958406e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:27.937895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b0487-572d-453b-af08-a5ac9958406e None admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:27.938170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b0487-572d-453b-af08-a5ac9958406e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:27.938437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b0487-572d-453b-af08-a5ac9958406e None admin] PATH_INFO: `/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/7db75c20e9744feb8b5dc78c554f4977/roles/6851de0f376f41a8a000506d1df67181` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:27.940697 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b0487-572d-453b-af08-a5ac9958406e None admin] RBAC: Authorizing `identity:create_grant(project_id=29b2e27932984de79ee8dfbe95312ac2, user_id=7db75c20e9744feb8b5dc78c554f4977, role_id=6851de0f376f41a8a000506d1df67181)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:27.942200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b0487-572d-453b-af08-a5ac9958406e None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:28.007717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 545/1084] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:25:27 2022] PUT /identity/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/7db75c20e9744feb8b5dc78c554f4977/roles/6851de0f376f41a8a000506d1df67181 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:25:29.632099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9464b53-a7d0-42eb-9ab7-c70229456f99 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:29.632740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9464b53-a7d0-42eb-9ab7-c70229456f99 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:29.632740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9464b53-a7d0-42eb-9ab7-c70229456f99 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:29.634045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 540/1085] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:29.639768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7637a3f-f8d1-4af1-b45f-6d014d25f030 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:29.640213 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7637a3f-f8d1-4af1-b45f-6d014d25f030 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:29.640318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7637a3f-f8d1-4af1-b45f-6d014d25f030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:29.658958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a7637a3f-f8d1-4af1-b45f-6d014d25f030 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:29.660170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a7637a3f-f8d1-4af1-b45f-6d014d25f030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:29.725620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 546/1086] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:29.993767 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9464b53-a7d0-42eb-9ab7-c70229456f99 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:30.099546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9464b53-a7d0-42eb-9ab7-c70229456f99 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:30.135939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a4393-e74b-4fc2-911a-be22974cc6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:30.136568 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a4393-e74b-4fc2-911a-be22974cc6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:30.136837 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a4393-e74b-4fc2-911a-be22974cc6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:30.137109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a4393-e74b-4fc2-911a-be22974cc6e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:30.153964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:30.156724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a4393-e74b-4fc2-911a-be22974cc6e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-433a4393-e74b-4fc2-911a-be22974cc6e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:25:30.161611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:30.163764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:30.163764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:30.163764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:30.163764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:30.163764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:25:30.163764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:25:30.163764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:25:30.163764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:25:30.163764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:25:30.163764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:25:30.163764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:25:30.163764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:30.163764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 541/1087] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:25:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:30.168759 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7637a3f-f8d1-4af1-b45f-6d014d25f030 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:30.187833 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84eed6-885b-4f23-b518-cfd45987048a None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:30.188323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84eed6-885b-4f23-b518-cfd45987048a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:30.188841 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84eed6-885b-4f23-b518-cfd45987048a None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:30.190500 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 542/1088] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:25:30.195652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b9b87-601f-44df-a68a-45a9b4294dfe None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:30.195927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b9b87-601f-44df-a68a-45a9b4294dfe None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:30.196190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b9b87-601f-44df-a68a-45a9b4294dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:30.214946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-5c9b9b87-601f-44df-a68a-45a9b4294dfe None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:30.216136 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5c9b9b87-601f-44df-a68a-45a9b4294dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:30.235586 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:30.270585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7637a3f-f8d1-4af1-b45f-6d014d25f030 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:30.285861 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 543/1089] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:30.304351 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1394c-b0e1-4412-bdde-3cb154088fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:30.304957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1394c-b0e1-4412-bdde-3cb154088fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:30.305221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1394c-b0e1-4412-bdde-3cb154088fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:30.305484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1394c-b0e1-4412-bdde-3cb154088fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:30.306169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1394c-b0e1-4412-bdde-3cb154088fc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:30.306466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1394c-b0e1-4412-bdde-3cb154088fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:30.307807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a1394c-b0e1-4412-bdde-3cb154088fc2 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:30.321356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 547/1090] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:25:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:30.322512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b9b87-601f-44df-a68a-45a9b4294dfe None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:30.417811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b9b87-601f-44df-a68a-45a9b4294dfe None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:30.445717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:30.451574 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca171afc-44e0-49ab-8e17-d46000be6620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:30.452186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca171afc-44e0-49ab-8e17-d46000be6620 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:30.452497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca171afc-44e0-49ab-8e17-d46000be6620 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:30.452762 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca171afc-44e0-49ab-8e17-d46000be6620 None admin] PATH_INFO: `/v3/users/dff0fc41ad6c4a288ca455180bd111fd` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:30.453943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca171afc-44e0-49ab-8e17-d46000be6620 None admin] RBAC: Authorizing `identity:get_user(user_id=dff0fc41ad6c4a288ca455180bd111fd)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:30.456761 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca171afc-44e0-49ab-8e17-d46000be6620 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:30.456942 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 544/1091] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:25:30 2022] GET /identity/v3/users/dff0fc41ad6c4a288ca455180bd111fd => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:30.462646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1394c-b0e1-4412-bdde-3cb154088fc2 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:30.565604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca171afc-44e0-49ab-8e17-d46000be6620 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:30.568639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1394c-b0e1-4412-bdde-3cb154088fc2 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:30.594399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:30.601831 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cae320-94ad-4a97-8aea-e0a81918ed6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:30.605483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cae320-94ad-4a97-8aea-e0a81918ed6b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:30.605483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cae320-94ad-4a97-8aea-e0a81918ed6b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:30.605483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cae320-94ad-4a97-8aea-e0a81918ed6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:30.666530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca171afc-44e0-49ab-8e17-d46000be6620 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:30.671279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cae320-94ad-4a97-8aea-e0a81918ed6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:30.672726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cae320-94ad-4a97-8aea-e0a81918ed6b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:30.701979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5dc4f-53bf-4642-883d-f1503a10d8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:30.702553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5dc4f-53bf-4642-883d-f1503a10d8ed None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:30.702822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5dc4f-53bf-4642-883d-f1503a10d8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:30.703081 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5dc4f-53bf-4642-883d-f1503a10d8ed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:30.712865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5dc4f-53bf-4642-883d-f1503a10d8ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-9ee5dc4f-53bf-4642-883d-f1503a10d8ed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:25:30.717669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:30.718700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:30.719549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:30.719549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:30.719549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:30.719549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:30.719549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:25:30.719549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:25:30.719549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:25:30.719549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:25:30.719549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:25:30.719549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:25:30.719549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 16 05:25:30.719549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:30.719549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 545/1092] 10.222.0.49 () {56 vars in 1120 bytes} [Fri Sep 16 05:25:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:30.724875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5dc4f-53bf-4642-883d-f1503a10d8ed None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:30.772804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 548/1093] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:25:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 16 05:25:30.779502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cae320-94ad-4a97-8aea-e0a81918ed6b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:30.784916 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:30.806836 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:31.004854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5dc4f-53bf-4642-883d-f1503a10d8ed None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:31.006484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cae320-94ad-4a97-8aea-e0a81918ed6b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:31.039688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a69cbb-3a11-48a1-825b-b6da293ab922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:31.040285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a69cbb-3a11-48a1-825b-b6da293ab922 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:31.040550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a69cbb-3a11-48a1-825b-b6da293ab922 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:31.040881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a69cbb-3a11-48a1-825b-b6da293ab922 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:31.041454 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4147a8-6766-4de3-a1a0-d030d3638f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:31.041589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a69cbb-3a11-48a1-825b-b6da293ab922 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:31.041744 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a69cbb-3a11-48a1-825b-b6da293ab922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:31.042111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4147a8-6766-4de3-a1a0-d030d3638f9b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:31.042384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4147a8-6766-4de3-a1a0-d030d3638f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:31.042650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4147a8-6766-4de3-a1a0-d030d3638f9b None admin] PATH_INFO: `/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:31.043112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a69cbb-3a11-48a1-825b-b6da293ab922 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:31.043817 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4147a8-6766-4de3-a1a0-d030d3638f9b None admin] RBAC: Authorizing `identity:get_project(project_id=3f951307f9bc405cbc7fca51f6b4fae1)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:31.045348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4147a8-6766-4de3-a1a0-d030d3638f9b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:31.052388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 549/1094] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:25:30 2022] GET /identity/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1 => generated 311 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:31.052536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 546/1095] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 16 05:25:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:31.053429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4147a8-6766-4de3-a1a0-d030d3638f9b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:31.056610 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a69cbb-3a11-48a1-825b-b6da293ab922 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:31.146499 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:31.152002 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4147a8-6766-4de3-a1a0-d030d3638f9b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:31.154580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a69cbb-3a11-48a1-825b-b6da293ab922 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:31.173885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:31.187506 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e84c59-a97e-4b53-ad04-47d3f03d991c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:31.187720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928a5c3-946b-4db1-95c7-bccd59d3d514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:31.188104 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e84c59-a97e-4b53-ad04-47d3f03d991c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:31.188383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928a5c3-946b-4db1-95c7-bccd59d3d514 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:31.188501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e84c59-a97e-4b53-ad04-47d3f03d991c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:31.188655 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928a5c3-946b-4db1-95c7-bccd59d3d514 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:31.188750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e84c59-a97e-4b53-ad04-47d3f03d991c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:31.188916 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928a5c3-946b-4db1-95c7-bccd59d3d514 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:31.189473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e84c59-a97e-4b53-ad04-47d3f03d991c None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=3f951307f9bc405cbc7fca51f6b4fae1, user.id=dff0fc41ad6c4a288ca455180bd111fd) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:31.189787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e84c59-a97e-4b53-ad04-47d3f03d991c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:31.191229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e84c59-a97e-4b53-ad04-47d3f03d991c None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:31.197072 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 550/1096] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:25:31 2022] GET /identity/v3/role_assignments?user.id=dff0fc41ad6c4a288ca455180bd111fd&scope.project.id=3f951307f9bc405cbc7fca51f6b4fae1&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:31.209749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928a5c3-946b-4db1-95c7-bccd59d3d514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:31.209749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928a5c3-946b-4db1-95c7-bccd59d3d514 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-a928a5c3-946b-4db1-95c7-bccd59d3d514 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:25:31.217031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:31.218221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:25:31.219135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:25:31.219135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:31.219135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:31.219135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 547/1097] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:31.224210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e84c59-a97e-4b53-ad04-47d3f03d991c None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:31.332043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e84c59-a97e-4b53-ad04-47d3f03d991c None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:31.366768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715b235-7a1e-4967-a2f7-1ef0ee505c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:31.367354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715b235-7a1e-4967-a2f7-1ef0ee505c9b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:31.367624 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715b235-7a1e-4967-a2f7-1ef0ee505c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:31.367886 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715b235-7a1e-4967-a2f7-1ef0ee505c9b None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:31.368522 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715b235-7a1e-4967-a2f7-1ef0ee505c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:31.368817 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715b235-7a1e-4967-a2f7-1ef0ee505c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:31.370199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715b235-7a1e-4967-a2f7-1ef0ee505c9b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:31.382840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 551/1098] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:31.389095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928a5c3-946b-4db1-95c7-bccd59d3d514 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:31.489261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928a5c3-946b-4db1-95c7-bccd59d3d514 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:31.522313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4248f-c3c9-4d15-874f-47c93e5d0acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:31.522938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4248f-c3c9-4d15-874f-47c93e5d0acb None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:31.523213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4248f-c3c9-4d15-874f-47c93e5d0acb None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:31.523479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4248f-c3c9-4d15-874f-47c93e5d0acb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:31.531386 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4248f-c3c9-4d15-874f-47c93e5d0acb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:31.532809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4248f-c3c9-4d15-874f-47c93e5d0acb None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-72d4248f-c3c9-4d15-874f-47c93e5d0acb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:25:31.543825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:31.544907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:31.545921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:31.545921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:31.545921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:25:31.545921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:25:31.545921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:25:31.545921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:25:31.545921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:25:31.545921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:25:31.545921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 16 05:25:31.545921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:31.545921 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 548/1099] 10.222.0.49 () {56 vars in 1093 bytes} [Fri Sep 16 05:25:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:31.550953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715b235-7a1e-4967-a2f7-1ef0ee505c9b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:31.573152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:31.650534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715b235-7a1e-4967-a2f7-1ef0ee505c9b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:31.683064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbe656-c6c0-4acf-9c05-db7c17d8e538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:31.683707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe656-c6c0-4acf-9c05-db7c17d8e538 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:31.683930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe656-c6c0-4acf-9c05-db7c17d8e538 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:31.684199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe656-c6c0-4acf-9c05-db7c17d8e538 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:31.685429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbe656-c6c0-4acf-9c05-db7c17d8e538 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:31.685732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbe656-c6c0-4acf-9c05-db7c17d8e538 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:31.687212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbe656-c6c0-4acf-9c05-db7c17d8e538 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:31.704621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 552/1100] 10.222.0.49 () {56 vars in 1133 bytes} [Fri Sep 16 05:25:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:31.710418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4248f-c3c9-4d15-874f-47c93e5d0acb None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:31.741393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:31.809098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4248f-c3c9-4d15-874f-47c93e5d0acb None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:31.846866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a559662-aabf-44ff-9e80-5078848a890e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:31.847463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a559662-aabf-44ff-9e80-5078848a890e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:31.847737 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a559662-aabf-44ff-9e80-5078848a890e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:31.848003 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a559662-aabf-44ff-9e80-5078848a890e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:31.913541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a559662-aabf-44ff-9e80-5078848a890e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:31.914920 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a559662-aabf-44ff-9e80-5078848a890e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:31.948896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:32.015967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 549/1101] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:25:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:25:32.022004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbe656-c6c0-4acf-9c05-db7c17d8e538 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:32.096735 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:32.118391 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbe656-c6c0-4acf-9c05-db7c17d8e538 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:32.154132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddb551-c8d8-4eb9-a7ea-9c1687460fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:32.154749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddb551-c8d8-4eb9-a7ea-9c1687460fff None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:32.155015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddb551-c8d8-4eb9-a7ea-9c1687460fff None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:32.155275 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddb551-c8d8-4eb9-a7ea-9c1687460fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:32.164906 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddb551-c8d8-4eb9-a7ea-9c1687460fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:32.166292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddb551-c8d8-4eb9-a7ea-9c1687460fff None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-f8ddb551-c8d8-4eb9-a7ea-9c1687460fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:25:32.181515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:32.182474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:25:32.183656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:25:32.183656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:25:32.183656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:32.183656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 553/1102] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:32.189073 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a559662-aabf-44ff-9e80-5078848a890e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:32.285146 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a559662-aabf-44ff-9e80-5078848a890e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:32.313424 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:32.318147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964359f-2780-47ee-a5b7-a3c6afc14893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:32.318737 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964359f-2780-47ee-a5b7-a3c6afc14893 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:32.319009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964359f-2780-47ee-a5b7-a3c6afc14893 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:32.319273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964359f-2780-47ee-a5b7-a3c6afc14893 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:32.319897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964359f-2780-47ee-a5b7-a3c6afc14893 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:32.320190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964359f-2780-47ee-a5b7-a3c6afc14893 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:32.321525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964359f-2780-47ee-a5b7-a3c6afc14893 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:32.333452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 550/1103] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:32.339001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddb551-c8d8-4eb9-a7ea-9c1687460fff None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:32.438437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddb551-c8d8-4eb9-a7ea-9c1687460fff None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:32.466615 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:32.472571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d1fe9-7d2d-41d9-90a0-9fff3d47fe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:32.473195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d1fe9-7d2d-41d9-90a0-9fff3d47fe03 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:32.476862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d1fe9-7d2d-41d9-90a0-9fff3d47fe03 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:32.476862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d1fe9-7d2d-41d9-90a0-9fff3d47fe03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:32.480198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d1fe9-7d2d-41d9-90a0-9fff3d47fe03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:32.481635 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d1fe9-7d2d-41d9-90a0-9fff3d47fe03 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-ce3d1fe9-7d2d-41d9-90a0-9fff3d47fe03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:25:32.493105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:32.494208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:32.495198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:25:32.495198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:25:32.495198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:25:32.495198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:25:32.495198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:25:32.495198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:25:32.495198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:25:32.495198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:32.495198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 554/1104] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:25:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:25:32.500338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964359f-2780-47ee-a5b7-a3c6afc14893 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:32.597729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964359f-2780-47ee-a5b7-a3c6afc14893 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:32.630171 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe543a0-8c3e-412e-a6d9-1c53ea76573f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:32.630766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe543a0-8c3e-412e-a6d9-1c53ea76573f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:32.631040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe543a0-8c3e-412e-a6d9-1c53ea76573f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:32.631306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe543a0-8c3e-412e-a6d9-1c53ea76573f None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:32.631985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe543a0-8c3e-412e-a6d9-1c53ea76573f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:32.632286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe543a0-8c3e-412e-a6d9-1c53ea76573f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:32.633690 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe543a0-8c3e-412e-a6d9-1c53ea76573f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:32.645675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 551/1105] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:25:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:32.651659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d1fe9-7d2d-41d9-90a0-9fff3d47fe03 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:32.748131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d1fe9-7d2d-41d9-90a0-9fff3d47fe03 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:32.780944 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e23e9a-3b77-4935-81dd-d3a86d6a85f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:32.781526 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e23e9a-3b77-4935-81dd-d3a86d6a85f4 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:32.781793 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e23e9a-3b77-4935-81dd-d3a86d6a85f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:32.782090 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e23e9a-3b77-4935-81dd-d3a86d6a85f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:32.782808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e23e9a-3b77-4935-81dd-d3a86d6a85f4 None admin] RBAC: Adding query filter params (role.id=6851de0f376f41a8a000506d1df67181, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=7db75c20e9744feb8b5dc78c554f4977) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:32.783095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e23e9a-3b77-4935-81dd-d3a86d6a85f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:32.784487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e23e9a-3b77-4935-81dd-d3a86d6a85f4 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:32.790150 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 555/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:25:32 2022] GET /identity/v3/role_assignments?user.id=7db75c20e9744feb8b5dc78c554f4977&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=6851de0f376f41a8a000506d1df67181 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:33.422058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98718d9-456c-4fec-aa9a-4f927532bdc2 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:33.422579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98718d9-456c-4fec-aa9a-4f927532bdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:33.422579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98718d9-456c-4fec-aa9a-4f927532bdc2 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:33.423720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 552/1107] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:33.429342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398a386-cb92-45de-935c-09cdbcac6381 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:33.429618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398a386-cb92-45de-935c-09cdbcac6381 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:33.429907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398a386-cb92-45de-935c-09cdbcac6381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:33.448670 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c398a386-cb92-45de-935c-09cdbcac6381 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:33.450180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c398a386-cb92-45de-935c-09cdbcac6381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:33.500745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:33.518253 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 556/1108] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:33.795792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98718d9-456c-4fec-aa9a-4f927532bdc2 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:33.848338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:33.894746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98718d9-456c-4fec-aa9a-4f927532bdc2 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:33.929383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2da410-165f-4aec-ba2c-ba59db22fe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:33.930033 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2da410-165f-4aec-ba2c-ba59db22fe50 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:33.930310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2da410-165f-4aec-ba2c-ba59db22fe50 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:33.930578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2da410-165f-4aec-ba2c-ba59db22fe50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:33.931672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2da410-165f-4aec-ba2c-ba59db22fe50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:33.933089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2da410-165f-4aec-ba2c-ba59db22fe50 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:33.934957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 553/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:33.940448 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398a386-cb92-45de-935c-09cdbcac6381 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:34.037133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398a386-cb92-45de-935c-09cdbcac6381 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:34.075335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222348a-a07b-4a0b-91ff-5c1f14c805e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:34.075962 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222348a-a07b-4a0b-91ff-5c1f14c805e7 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:34.076238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222348a-a07b-4a0b-91ff-5c1f14c805e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:34.076507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222348a-a07b-4a0b-91ff-5c1f14c805e7 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:34.077159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222348a-a07b-4a0b-91ff-5c1f14c805e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:34.078605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222348a-a07b-4a0b-91ff-5c1f14c805e7 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:34.174089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 557/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:25:33 2022] POST /identity/v3/users => generated 286 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:25:36.339419 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804b025-1bf8-4e6d-89bc-8966c0720196 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:36.339946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804b025-1bf8-4e6d-89bc-8966c0720196 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:36.339946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804b025-1bf8-4e6d-89bc-8966c0720196 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:36.341804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 554/1111] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:36.347076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901bfb2f-a9c5-462a-846c-6eabf06c2609 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:36.347467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901bfb2f-a9c5-462a-846c-6eabf06c2609 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:36.347613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901bfb2f-a9c5-462a-846c-6eabf06c2609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:36.365266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-901bfb2f-a9c5-462a-846c-6eabf06c2609 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:36.366447 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-901bfb2f-a9c5-462a-846c-6eabf06c2609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:36.399154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:36.429636 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 558/1112] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:36.695782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804b025-1bf8-4e6d-89bc-8966c0720196 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:36.796301 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804b025-1bf8-4e6d-89bc-8966c0720196 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:36.815425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:36.828441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c1e97-a3e3-427c-a16e-7fb56870c250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:36.829312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c1e97-a3e3-427c-a16e-7fb56870c250 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:36.829423 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c1e97-a3e3-427c-a16e-7fb56870c250 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:36.829605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c1e97-a3e3-427c-a16e-7fb56870c250 None admin] PATH_INFO: `/v3/roles/93f699f03f0f41498415ab9533bdfcb4` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:36.830980 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c1e97-a3e3-427c-a16e-7fb56870c250 None admin] RBAC: Authorizing `identity:get_role(role_id=93f699f03f0f41498415ab9533bdfcb4)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:36.832314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c1e97-a3e3-427c-a16e-7fb56870c250 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:36.833589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 555/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:25:36 2022] GET /identity/v3/roles/93f699f03f0f41498415ab9533bdfcb4 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:36.839360 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901bfb2f-a9c5-462a-846c-6eabf06c2609 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:36.936266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901bfb2f-a9c5-462a-846c-6eabf06c2609 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:36.968505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f704da0-068a-4b6f-bf05-4e586c2809ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:36.969049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f704da0-068a-4b6f-bf05-4e586c2809ee None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:36.969316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f704da0-068a-4b6f-bf05-4e586c2809ee None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:36.969578 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f704da0-068a-4b6f-bf05-4e586c2809ee None admin] PATH_INFO: `/v3/users/6acd9771263a46dcb367b5826cdc10fa` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:36.982776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f704da0-068a-4b6f-bf05-4e586c2809ee None admin] RBAC: Authorizing `identity:get_user(user_id=6acd9771263a46dcb367b5826cdc10fa)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:36.984185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f704da0-068a-4b6f-bf05-4e586c2809ee None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:36.985751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 559/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:25:36 2022] GET /identity/v3/users/6acd9771263a46dcb367b5826cdc10fa => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:36.991663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c1e97-a3e3-427c-a16e-7fb56870c250 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:37.087765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c1e97-a3e3-427c-a16e-7fb56870c250 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:37.119729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f734ae-6e44-48c1-9f9e-bc14a90d99bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:37.120319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f734ae-6e44-48c1-9f9e-bc14a90d99bf None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:37.120588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f734ae-6e44-48c1-9f9e-bc14a90d99bf None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:37.120849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f734ae-6e44-48c1-9f9e-bc14a90d99bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:37.174077 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:37.186316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f734ae-6e44-48c1-9f9e-bc14a90d99bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:37.187733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f734ae-6e44-48c1-9f9e-bc14a90d99bf None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:37.281678 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 556/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:25:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 16 05:25:37.287579 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f704da0-068a-4b6f-bf05-4e586c2809ee None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:37.314985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:37.383232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f704da0-068a-4b6f-bf05-4e586c2809ee None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:37.415430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bf687-7eb6-4260-af39-b6ab83f7d284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:37.416034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bf687-7eb6-4260-af39-b6ab83f7d284 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:37.416304 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bf687-7eb6-4260-af39-b6ab83f7d284 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:37.416569 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bf687-7eb6-4260-af39-b6ab83f7d284 None admin] PATH_INFO: `/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:37.417640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bf687-7eb6-4260-af39-b6ab83f7d284 None admin] RBAC: Authorizing `identity:get_project(project_id=3f951307f9bc405cbc7fca51f6b4fae1)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:37.419601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947bf687-7eb6-4260-af39-b6ab83f7d284 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:37.421200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 560/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:25:37 2022] GET /identity/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:37.427215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f734ae-6e44-48c1-9f9e-bc14a90d99bf None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:37.510543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:37.522106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f734ae-6e44-48c1-9f9e-bc14a90d99bf None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:37.555483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16cb89-1d76-415c-b2fb-5282fbe65111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:37.556005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16cb89-1d76-415c-b2fb-5282fbe65111 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:37.556278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16cb89-1d76-415c-b2fb-5282fbe65111 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:37.556544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16cb89-1d76-415c-b2fb-5282fbe65111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:37.557257 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16cb89-1d76-415c-b2fb-5282fbe65111 None admin] RBAC: Adding query filter params (role.id=93f699f03f0f41498415ab9533bdfcb4, scope.project.id=3f951307f9bc405cbc7fca51f6b4fae1, user.id=6acd9771263a46dcb367b5826cdc10fa) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:37.557559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16cb89-1d76-415c-b2fb-5282fbe65111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:37.559001 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16cb89-1d76-415c-b2fb-5282fbe65111 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:37.564203 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 557/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:25:37 2022] GET /identity/v3/role_assignments?user.id=6acd9771263a46dcb367b5826cdc10fa&scope.project.id=3f951307f9bc405cbc7fca51f6b4fae1&role.id=93f699f03f0f41498415ab9533bdfcb4 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:39.719017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d1970-929d-4332-af94-169f3414ecff None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:39.719546 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d1970-929d-4332-af94-169f3414ecff None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:39.719546 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d1970-929d-4332-af94-169f3414ecff None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:39.720879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 561/1118] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:39.726186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed46aa5-2e2c-494a-bf68-a01855eb51cf None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:39.726606 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed46aa5-2e2c-494a-bf68-a01855eb51cf None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:39.726709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed46aa5-2e2c-494a-bf68-a01855eb51cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:39.744511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-4ed46aa5-2e2c-494a-bf68-a01855eb51cf None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:39.745661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4ed46aa5-2e2c-494a-bf68-a01855eb51cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:39.810333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 558/1119] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:25:40.079828 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d1970-929d-4332-af94-169f3414ecff None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:40.180846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:40.188356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d1970-929d-4332-af94-169f3414ecff None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:40.221365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c3aba-3e68-49ec-a96c-89a3cb8e529e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:40.222024 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c3aba-3e68-49ec-a96c-89a3cb8e529e None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:40.222301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c3aba-3e68-49ec-a96c-89a3cb8e529e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:40.222570 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c3aba-3e68-49ec-a96c-89a3cb8e529e None admin] PATH_INFO: `/v3/roles/93f699f03f0f41498415ab9533bdfcb4` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:40.223976 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c3aba-3e68-49ec-a96c-89a3cb8e529e None admin] RBAC: Authorizing `identity:get_role(role_id=93f699f03f0f41498415ab9533bdfcb4)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:40.225336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c3aba-3e68-49ec-a96c-89a3cb8e529e None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:40.226686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 562/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:25:40 2022] GET /identity/v3/roles/93f699f03f0f41498415ab9533bdfcb4 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:40.233162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed46aa5-2e2c-494a-bf68-a01855eb51cf None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:40.318946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:40.336220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed46aa5-2e2c-494a-bf68-a01855eb51cf None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:40.369343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509aa3c8-c6fc-4102-89cb-7509cb7bacf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:40.369988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509aa3c8-c6fc-4102-89cb-7509cb7bacf9 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:40.370261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509aa3c8-c6fc-4102-89cb-7509cb7bacf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:40.370524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509aa3c8-c6fc-4102-89cb-7509cb7bacf9 None admin] PATH_INFO: `/v3/users/6acd9771263a46dcb367b5826cdc10fa` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:40.371612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509aa3c8-c6fc-4102-89cb-7509cb7bacf9 None admin] RBAC: Authorizing `identity:get_user(user_id=6acd9771263a46dcb367b5826cdc10fa)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:40.373344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509aa3c8-c6fc-4102-89cb-7509cb7bacf9 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:40.374918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 559/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:25:40 2022] GET /identity/v3/users/6acd9771263a46dcb367b5826cdc10fa => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:25:40.380682 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c3aba-3e68-49ec-a96c-89a3cb8e529e None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:40.475688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c3aba-3e68-49ec-a96c-89a3cb8e529e None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:40.508023 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbe87a-379a-443b-8e9a-53d38aaf8d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:40.508697 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbe87a-379a-443b-8e9a-53d38aaf8d4f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:40.508964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbe87a-379a-443b-8e9a-53d38aaf8d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:40.509222 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbe87a-379a-443b-8e9a-53d38aaf8d4f None admin] PATH_INFO: `/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:40.510316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fbe87a-379a-443b-8e9a-53d38aaf8d4f None admin] RBAC: Authorizing `identity:get_project(project_id=3f951307f9bc405cbc7fca51f6b4fae1)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:40.511715 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fbe87a-379a-443b-8e9a-53d38aaf8d4f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:40.513328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 563/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:25:40 2022] GET /identity/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:40.519466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509aa3c8-c6fc-4102-89cb-7509cb7bacf9 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:40.621221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509aa3c8-c6fc-4102-89cb-7509cb7bacf9 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:40.653853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabe435-d766-4af7-aefd-54402fc908a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:40.654467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabe435-d766-4af7-aefd-54402fc908a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:40.654730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabe435-d766-4af7-aefd-54402fc908a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:40.654993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabe435-d766-4af7-aefd-54402fc908a9 None admin] PATH_INFO: `/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1/users/6acd9771263a46dcb367b5826cdc10fa/roles/93f699f03f0f41498415ab9533bdfcb4` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:40.657072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabe435-d766-4af7-aefd-54402fc908a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f951307f9bc405cbc7fca51f6b4fae1, user_id=6acd9771263a46dcb367b5826cdc10fa, role_id=93f699f03f0f41498415ab9533bdfcb4)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:40.658507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabe435-d766-4af7-aefd-54402fc908a9 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:40.707220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 560/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:25:40 2022] PUT /identity/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1/users/6acd9771263a46dcb367b5826cdc10fa/roles/93f699f03f0f41498415ab9533bdfcb4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:25:42.810564 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad83983-b845-4588-9332-5be524edfcc3 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:42.811147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad83983-b845-4588-9332-5be524edfcc3 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:42.811395 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad83983-b845-4588-9332-5be524edfcc3 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:42.813087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 564/1124] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:42.818493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbda5f0-4832-4e08-a70c-4f7467e3ab91 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:42.818902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbda5f0-4832-4e08-a70c-4f7467e3ab91 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:42.819036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbda5f0-4832-4e08-a70c-4f7467e3ab91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:42.837306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-2dbda5f0-4832-4e08-a70c-4f7467e3ab91 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:42.838521 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2dbda5f0-4832-4e08-a70c-4f7467e3ab91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:42.841344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:42.907900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 561/1125] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:43.179836 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad83983-b845-4588-9332-5be524edfcc3 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:43.210358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:43.277732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad83983-b845-4588-9332-5be524edfcc3 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:43.310893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b54a12-a097-44f6-9ad2-abf17bee7669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:43.311499 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b54a12-a097-44f6-9ad2-abf17bee7669 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:43.311768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b54a12-a097-44f6-9ad2-abf17bee7669 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:43.312028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b54a12-a097-44f6-9ad2-abf17bee7669 None admin] PATH_INFO: `/v3/roles/93f699f03f0f41498415ab9533bdfcb4` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:43.313371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b54a12-a097-44f6-9ad2-abf17bee7669 None admin] RBAC: Authorizing `identity:get_role(role_id=93f699f03f0f41498415ab9533bdfcb4)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:43.314734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b54a12-a097-44f6-9ad2-abf17bee7669 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:43.316028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 565/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:25:43 2022] GET /identity/v3/roles/93f699f03f0f41498415ab9533bdfcb4 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:43.321958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbda5f0-4832-4e08-a70c-4f7467e3ab91 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:43.417522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbda5f0-4832-4e08-a70c-4f7467e3ab91 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:43.450509 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ad1e1-8d21-4459-9e58-dbb2c510307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:43.451215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad1e1-8d21-4459-9e58-dbb2c510307e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:43.451493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad1e1-8d21-4459-9e58-dbb2c510307e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:43.451764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ad1e1-8d21-4459-9e58-dbb2c510307e None admin] PATH_INFO: `/v3/users/6acd9771263a46dcb367b5826cdc10fa` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:43.452873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ad1e1-8d21-4459-9e58-dbb2c510307e None admin] RBAC: Authorizing `identity:get_user(user_id=6acd9771263a46dcb367b5826cdc10fa)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:43.454322 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ad1e1-8d21-4459-9e58-dbb2c510307e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:43.455852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 562/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:25:43 2022] GET /identity/v3/users/6acd9771263a46dcb367b5826cdc10fa => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:43.461499 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b54a12-a097-44f6-9ad2-abf17bee7669 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:43.554401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:43.557184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b54a12-a097-44f6-9ad2-abf17bee7669 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:43.588826 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243002fb-6421-459b-a8ce-1f4c3bf45690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:43.589407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243002fb-6421-459b-a8ce-1f4c3bf45690 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:43.589676 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243002fb-6421-459b-a8ce-1f4c3bf45690 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:43.589966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243002fb-6421-459b-a8ce-1f4c3bf45690 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:43.655245 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243002fb-6421-459b-a8ce-1f4c3bf45690 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:43.656646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243002fb-6421-459b-a8ce-1f4c3bf45690 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:43.745549 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:43.748993 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 566/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:25:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:25:43.754950 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ad1e1-8d21-4459-9e58-dbb2c510307e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:43.765956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:43.853760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ad1e1-8d21-4459-9e58-dbb2c510307e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:43.886565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7d0bf-990d-4fa3-82fc-0822e18f971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:43.887178 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7d0bf-990d-4fa3-82fc-0822e18f971f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:43.887947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7d0bf-990d-4fa3-82fc-0822e18f971f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:43.888377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7d0bf-990d-4fa3-82fc-0822e18f971f None admin] PATH_INFO: `/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:43.889690 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7d0bf-990d-4fa3-82fc-0822e18f971f None admin] RBAC: Authorizing `identity:get_project(project_id=3f951307f9bc405cbc7fca51f6b4fae1)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:43.891132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7d0bf-990d-4fa3-82fc-0822e18f971f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:43.892973 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 563/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:25:43 2022] GET /identity/v3/projects/3f951307f9bc405cbc7fca51f6b4fae1 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:43.899127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243002fb-6421-459b-a8ce-1f4c3bf45690 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:43.995524 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243002fb-6421-459b-a8ce-1f4c3bf45690 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:44.029869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6628f-d2a8-4a15-b758-e5bcba08baf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:44.030401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6628f-d2a8-4a15-b758-e5bcba08baf3 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:44.030675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6628f-d2a8-4a15-b758-e5bcba08baf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:44.030941 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6628f-d2a8-4a15-b758-e5bcba08baf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:44.031655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6628f-d2a8-4a15-b758-e5bcba08baf3 None admin] RBAC: Adding query filter params (role.id=93f699f03f0f41498415ab9533bdfcb4, scope.project.id=3f951307f9bc405cbc7fca51f6b4fae1, user.id=6acd9771263a46dcb367b5826cdc10fa) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:44.031961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6628f-d2a8-4a15-b758-e5bcba08baf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:44.033366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6628f-d2a8-4a15-b758-e5bcba08baf3 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:44.039789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 567/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:25:43 2022] GET /identity/v3/role_assignments?user.id=6acd9771263a46dcb367b5826cdc10fa&scope.project.id=3f951307f9bc405cbc7fca51f6b4fae1&role.id=93f699f03f0f41498415ab9533bdfcb4 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:46.204434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce7d4d-5244-409e-bb06-7ffc5a155131 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:46.204952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce7d4d-5244-409e-bb06-7ffc5a155131 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:46.204952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce7d4d-5244-409e-bb06-7ffc5a155131 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:46.206216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 564/1131] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:46.211489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a364bf-b94b-4c17-8208-ded8c818f2e0 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:46.211935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a364bf-b94b-4c17-8208-ded8c818f2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:46.212050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a364bf-b94b-4c17-8208-ded8c818f2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:46.229988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-91a364bf-b94b-4c17-8208-ded8c818f2e0 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:46.231688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-91a364bf-b94b-4c17-8208-ded8c818f2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:46.289284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:46.298132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 568/1132] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:46.572004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce7d4d-5244-409e-bb06-7ffc5a155131 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:46.646770 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:46.670469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce7d4d-5244-409e-bb06-7ffc5a155131 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:46.704584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055fe41a-0d63-4cd4-867b-5ffd89bbbef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:46.705191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fe41a-0d63-4cd4-867b-5ffd89bbbef1 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:46.705464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fe41a-0d63-4cd4-867b-5ffd89bbbef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:46.705728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055fe41a-0d63-4cd4-867b-5ffd89bbbef1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:46.706860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055fe41a-0d63-4cd4-867b-5ffd89bbbef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:46.708275 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055fe41a-0d63-4cd4-867b-5ffd89bbbef1 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:46.709815 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 565/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:46.716283 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a364bf-b94b-4c17-8208-ded8c818f2e0 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:46.810930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a364bf-b94b-4c17-8208-ded8c818f2e0 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:46.844054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f1939-a070-4def-b84b-94d8fa740ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:46.844582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f1939-a070-4def-b84b-94d8fa740ce1 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:46.844856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f1939-a070-4def-b84b-94d8fa740ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:46.845117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f1939-a070-4def-b84b-94d8fa740ce1 None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:46.845765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f1939-a070-4def-b84b-94d8fa740ce1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:46.847210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f1939-a070-4def-b84b-94d8fa740ce1 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:46.951004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 569/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:25:46 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:25:49.152318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c5d4a-1d4e-419f-8425-397cbda5ca5e None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:49.152840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c5d4a-1d4e-419f-8425-397cbda5ca5e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:49.152840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c5d4a-1d4e-419f-8425-397cbda5ca5e None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:49.154289 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 566/1135] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:49.159893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294bb2c-6839-475e-8b7b-41c0c15d6581 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:49.160346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294bb2c-6839-475e-8b7b-41c0c15d6581 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:49.160437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294bb2c-6839-475e-8b7b-41c0c15d6581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:49.178754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-b294bb2c-6839-475e-8b7b-41c0c15d6581 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:49.179869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b294bb2c-6839-475e-8b7b-41c0c15d6581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:49.220373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:49.244170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 570/1136] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:49.524054 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c5d4a-1d4e-419f-8425-397cbda5ca5e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:49.622042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c5d4a-1d4e-419f-8425-397cbda5ca5e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:49.655293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51177f74-404a-4126-8248-623296a462d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:49.655901 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51177f74-404a-4126-8248-623296a462d0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:49.656175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51177f74-404a-4126-8248-623296a462d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:49.656441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51177f74-404a-4126-8248-623296a462d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:49.657569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51177f74-404a-4126-8248-623296a462d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:49.659007 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51177f74-404a-4126-8248-623296a462d0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:49.660556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 567/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:25:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:49.666488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294bb2c-6839-475e-8b7b-41c0c15d6581 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:49.761801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294bb2c-6839-475e-8b7b-41c0c15d6581 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:49.795137 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec329214-5658-4b11-9a22-58673a0f2757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:49.795666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec329214-5658-4b11-9a22-58673a0f2757 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:49.795938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec329214-5658-4b11-9a22-58673a0f2757 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:49.796202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec329214-5658-4b11-9a22-58673a0f2757 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:49.796835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec329214-5658-4b11-9a22-58673a0f2757 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:49.798278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec329214-5658-4b11-9a22-58673a0f2757 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:49.893966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 571/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:25:49 2022] POST /identity/v3/users => generated 286 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:25:52.092214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6e66c-49c2-43f6-b022-7d54bb81aff3 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:52.092759 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6e66c-49c2-43f6-b022-7d54bb81aff3 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:52.092759 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb6e66c-49c2-43f6-b022-7d54bb81aff3 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:52.094109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 568/1139] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:25:52.099471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84d3d0-6907-4a2b-ba09-a7e41688a1d0 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:52.099844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84d3d0-6907-4a2b-ba09-a7e41688a1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:52.100008 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84d3d0-6907-4a2b-ba09-a7e41688a1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:52.118414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-5a84d3d0-6907-4a2b-ba09-a7e41688a1d0 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:52.119596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5a84d3d0-6907-4a2b-ba09-a7e41688a1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:52.145088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:52.184027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 572/1140] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:52.453771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6e66c-49c2-43f6-b022-7d54bb81aff3 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:52.469005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:52.554249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb6e66c-49c2-43f6-b022-7d54bb81aff3 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:52.586405 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293dc65-4e0c-4ead-a2e8-44d7d54903db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:52.587014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293dc65-4e0c-4ead-a2e8-44d7d54903db None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:52.587283 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293dc65-4e0c-4ead-a2e8-44d7d54903db None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:52.587544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293dc65-4e0c-4ead-a2e8-44d7d54903db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:52.598024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d293dc65-4e0c-4ead-a2e8-44d7d54903db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-d293dc65-4e0c-4ead-a2e8-44d7d54903db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:25:52.602722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:52.603636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:52.606527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:52.606527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:52.606527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:52.606527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:52.606527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:25:52.606527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:25:52.606527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:25:52.606527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:25:52.606527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:25:52.606527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:25:52.606527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:25:52.606527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:52.606527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 569/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:25:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:52.610418 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84d3d0-6907-4a2b-ba09-a7e41688a1d0 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:52.706262 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84d3d0-6907-4a2b-ba09-a7e41688a1d0 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:52.739532 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8348d-4982-4553-9946-98c6c4e8553a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:52.740126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8348d-4982-4553-9946-98c6c4e8553a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:52.740403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8348d-4982-4553-9946-98c6c4e8553a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:52.740669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8348d-4982-4553-9946-98c6c4e8553a None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:52.741314 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8348d-4982-4553-9946-98c6c4e8553a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:52.741607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8348d-4982-4553-9946-98c6c4e8553a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:52.742993 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8348d-4982-4553-9946-98c6c4e8553a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:52.751001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 573/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:25:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:52.757025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293dc65-4e0c-4ead-a2e8-44d7d54903db None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:52.826536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:52.854469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293dc65-4e0c-4ead-a2e8-44d7d54903db None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:52.886709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2db86-2ba3-4fd3-ad7c-0271b7c1f177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:52.887819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2db86-2ba3-4fd3-ad7c-0271b7c1f177 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:52.888088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2db86-2ba3-4fd3-ad7c-0271b7c1f177 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:52.888348 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2db86-2ba3-4fd3-ad7c-0271b7c1f177 None admin] PATH_INFO: `/v3/users/61d02255d58847ddbc5168e872d6cf80` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:52.900985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2db86-2ba3-4fd3-ad7c-0271b7c1f177 None admin] RBAC: Authorizing `identity:get_user(user_id=61d02255d58847ddbc5168e872d6cf80)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:52.902514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2db86-2ba3-4fd3-ad7c-0271b7c1f177 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:52.904640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 570/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:25:52 2022] GET /identity/v3/users/61d02255d58847ddbc5168e872d6cf80 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:52.910559 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8348d-4982-4553-9946-98c6c4e8553a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:52.935461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:53.017854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8348d-4982-4553-9946-98c6c4e8553a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:53.050513 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a37058-01cc-4501-8878-93bcc6045b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:53.051101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a37058-01cc-4501-8878-93bcc6045b46 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:53.051374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a37058-01cc-4501-8878-93bcc6045b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:53.051640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a37058-01cc-4501-8878-93bcc6045b46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:53.116157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a37058-01cc-4501-8878-93bcc6045b46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:53.117588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a37058-01cc-4501-8878-93bcc6045b46 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:53.141518 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:53.214006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 574/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:25:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:25:53.219906 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2db86-2ba3-4fd3-ad7c-0271b7c1f177 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:53.319257 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2db86-2ba3-4fd3-ad7c-0271b7c1f177 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:53.347262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:53.352779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be47e1-1167-4faf-ab7c-5b1699d849be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:53.353394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be47e1-1167-4faf-ab7c-5b1699d849be None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:53.353665 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be47e1-1167-4faf-ab7c-5b1699d849be None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:53.353957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be47e1-1167-4faf-ab7c-5b1699d849be None admin] PATH_INFO: `/v3/projects/da2d23cd274c4b639a7a2aecdc4db1cb` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:53.355051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be47e1-1167-4faf-ab7c-5b1699d849be None admin] RBAC: Authorizing `identity:get_project(project_id=da2d23cd274c4b639a7a2aecdc4db1cb)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:53.356962 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be47e1-1167-4faf-ab7c-5b1699d849be None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:53.358572 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 571/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:25:53 2022] GET /identity/v3/projects/da2d23cd274c4b639a7a2aecdc4db1cb => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:53.364596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a37058-01cc-4501-8878-93bcc6045b46 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:53.461668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a37058-01cc-4501-8878-93bcc6045b46 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:53.484717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:53.495132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65841b4-735c-4c62-ab5e-297a35011016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:53.495721 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65841b4-735c-4c62-ab5e-297a35011016 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:53.495996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65841b4-735c-4c62-ab5e-297a35011016 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:53.496266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65841b4-735c-4c62-ab5e-297a35011016 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:53.497581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65841b4-735c-4c62-ab5e-297a35011016 None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=da2d23cd274c4b639a7a2aecdc4db1cb, user.id=61d02255d58847ddbc5168e872d6cf80) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:53.497912 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65841b4-735c-4c62-ab5e-297a35011016 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:53.499326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65841b4-735c-4c62-ab5e-297a35011016 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:53.504728 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 575/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:25:53 2022] GET /identity/v3/role_assignments?user.id=61d02255d58847ddbc5168e872d6cf80&scope.project.id=da2d23cd274c4b639a7a2aecdc4db1cb&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:55.628182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5897a3-91df-4689-b78e-8787d531234c None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:55.630489 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5897a3-91df-4689-b78e-8787d531234c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:55.630489 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5897a3-91df-4689-b78e-8787d531234c None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:55.630489 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 572/1147] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:55.635198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f940d-2107-4137-abf2-12e564f4c28f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:55.635570 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f940d-2107-4137-abf2-12e564f4c28f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:55.635733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291f940d-2107-4137-abf2-12e564f4c28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:55.654396 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-291f940d-2107-4137-abf2-12e564f4c28f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:55.655554 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-291f940d-2107-4137-abf2-12e564f4c28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:55.719376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 576/1148] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:56.007531 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5897a3-91df-4689-b78e-8787d531234c None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:56.106452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5897a3-91df-4689-b78e-8787d531234c None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:56.139653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12600ca-b098-4a2c-8197-53bc3687fc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:56.140269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12600ca-b098-4a2c-8197-53bc3687fc86 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:56.140542 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12600ca-b098-4a2c-8197-53bc3687fc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:56.140810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12600ca-b098-4a2c-8197-53bc3687fc86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:56.150841 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12600ca-b098-4a2c-8197-53bc3687fc86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-b12600ca-b098-4a2c-8197-53bc3687fc86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:25:56.156311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:56.157408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:56.158488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:56.158488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:56.158488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:56.158488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:56.158488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:25:56.158488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:25:56.158488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:25:56.158488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:25:56.158488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:25:56.158488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:25:56.158488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:25:56.158488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:25:56.158488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 573/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:56.164031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f940d-2107-4137-abf2-12e564f4c28f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:56.257910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291f940d-2107-4137-abf2-12e564f4c28f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:56.274496 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:56.290504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da46b13-8438-4516-b821-bea84a59a149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:56.293659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da46b13-8438-4516-b821-bea84a59a149 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:56.293659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da46b13-8438-4516-b821-bea84a59a149 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:56.293659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da46b13-8438-4516-b821-bea84a59a149 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:56.293659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da46b13-8438-4516-b821-bea84a59a149 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:56.293659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da46b13-8438-4516-b821-bea84a59a149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:56.294080 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da46b13-8438-4516-b821-bea84a59a149 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:56.301562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 577/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:25:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:56.307467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12600ca-b098-4a2c-8197-53bc3687fc86 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:56.363619 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:56.404691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12600ca-b098-4a2c-8197-53bc3687fc86 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:56.438265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ad9b5-0d13-4cc2-a864-a661c4d5c9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:56.438868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ad9b5-0d13-4cc2-a864-a661c4d5c9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:56.439141 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ad9b5-0d13-4cc2-a864-a661c4d5c9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:56.439404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180ad9b5-0d13-4cc2-a864-a661c4d5c9c2 None admin] PATH_INFO: `/v3/users/61d02255d58847ddbc5168e872d6cf80` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:56.440500 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ad9b5-0d13-4cc2-a864-a661c4d5c9c2 None admin] RBAC: Authorizing `identity:get_user(user_id=61d02255d58847ddbc5168e872d6cf80)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:56.441944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180ad9b5-0d13-4cc2-a864-a661c4d5c9c2 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:56.443475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 574/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:25:56 2022] GET /identity/v3/users/61d02255d58847ddbc5168e872d6cf80 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:56.449159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da46b13-8438-4516-b821-bea84a59a149 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:56.545369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da46b13-8438-4516-b821-bea84a59a149 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:56.577302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b0147-f0a4-4eba-af3e-88560b680e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:56.577933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b0147-f0a4-4eba-af3e-88560b680e19 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:56.578207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b0147-f0a4-4eba-af3e-88560b680e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:56.578467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b0147-f0a4-4eba-af3e-88560b680e19 None admin] PATH_INFO: `/v3/projects/da2d23cd274c4b639a7a2aecdc4db1cb` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:56.579540 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b0147-f0a4-4eba-af3e-88560b680e19 None admin] RBAC: Authorizing `identity:get_project(project_id=da2d23cd274c4b639a7a2aecdc4db1cb)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:56.580961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b0147-f0a4-4eba-af3e-88560b680e19 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:56.582564 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 578/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:25:56 2022] GET /identity/v3/projects/da2d23cd274c4b639a7a2aecdc4db1cb => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:56.588353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ad9b5-0d13-4cc2-a864-a661c4d5c9c2 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:56.683820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180ad9b5-0d13-4cc2-a864-a661c4d5c9c2 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:56.708827 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:56.717121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce191fc0-b554-45e3-b7e2-a44d98277771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:56.721771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce191fc0-b554-45e3-b7e2-a44d98277771 None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:56.721771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce191fc0-b554-45e3-b7e2-a44d98277771 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:56.721771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce191fc0-b554-45e3-b7e2-a44d98277771 None admin] PATH_INFO: `/v3/projects/da2d23cd274c4b639a7a2aecdc4db1cb/users/61d02255d58847ddbc5168e872d6cf80/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:56.721771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce191fc0-b554-45e3-b7e2-a44d98277771 None admin] RBAC: Authorizing `identity:create_grant(project_id=da2d23cd274c4b639a7a2aecdc4db1cb, user_id=61d02255d58847ddbc5168e872d6cf80, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:56.721771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce191fc0-b554-45e3-b7e2-a44d98277771 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:56.811708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 575/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:25:56 2022] PUT /identity/v3/projects/da2d23cd274c4b639a7a2aecdc4db1cb/users/61d02255d58847ddbc5168e872d6cf80/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:25:59.061128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5af2a-fcde-43fa-a836-1fb37815f124 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:59.061655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5af2a-fcde-43fa-a836-1fb37815f124 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:59.061655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5af2a-fcde-43fa-a836-1fb37815f124 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:59.063007 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 579/1154] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:25:59.068295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced165e-5ea1-4373-b61f-1d409f5f95da None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:59.068713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced165e-5ea1-4373-b61f-1d409f5f95da None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:59.068804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced165e-5ea1-4373-b61f-1d409f5f95da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:59.086593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-dced165e-5ea1-4373-b61f-1d409f5f95da None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:25:59.087743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-dced165e-5ea1-4373-b61f-1d409f5f95da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:25:59.154306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 576/1155] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:25:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:25:59.427769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5af2a-fcde-43fa-a836-1fb37815f124 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:59.477164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:59.532161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5af2a-fcde-43fa-a836-1fb37815f124 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:59.564577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3ee3e-144f-49a8-ab64-ab0aa1aafba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:59.565184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3ee3e-144f-49a8-ab64-ab0aa1aafba1 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:59.565453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3ee3e-144f-49a8-ab64-ab0aa1aafba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:59.565716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3ee3e-144f-49a8-ab64-ab0aa1aafba1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:59.576310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3ee3e-144f-49a8-ab64-ab0aa1aafba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-49a3ee3e-144f-49a8-ab64-ab0aa1aafba1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:25:59.581070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:25:59.582278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:25:59.583257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:25:59.583257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:25:59.583257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:25:59.583257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:25:59.583257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:25:59.583257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:25:59.583257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:25:59.583257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:25:59.583257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:25:59.583257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:25:59.583257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:25:59.583257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:25:59.583257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 580/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:25:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:25:59.588383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced165e-5ea1-4373-b61f-1d409f5f95da None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:59.683052 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced165e-5ea1-4373-b61f-1d409f5f95da None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:59.698909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:59.716210 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f67e8-4290-4688-a387-4f3a822b479f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:59.716792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f67e8-4290-4688-a387-4f3a822b479f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:59.717061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f67e8-4290-4688-a387-4f3a822b479f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:59.717323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f67e8-4290-4688-a387-4f3a822b479f None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:59.717990 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f67e8-4290-4688-a387-4f3a822b479f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:25:59.718282 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f67e8-4290-4688-a387-4f3a822b479f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:59.719615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f67e8-4290-4688-a387-4f3a822b479f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:59.727265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 577/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:25:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:59.733881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3ee3e-144f-49a8-ab64-ab0aa1aafba1 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:59.838858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3ee3e-144f-49a8-ab64-ab0aa1aafba1 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:25:59.856662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:25:59.872474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bd69d-166f-461c-891b-711a59d152c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:25:59.873087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bd69d-166f-461c-891b-711a59d152c5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:25:59.873379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bd69d-166f-461c-891b-711a59d152c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:25:59.873621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bd69d-166f-461c-891b-711a59d152c5 None admin] PATH_INFO: `/v3/users/61d02255d58847ddbc5168e872d6cf80` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:25:59.874733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bd69d-166f-461c-891b-711a59d152c5 None admin] RBAC: Authorizing `identity:get_user(user_id=61d02255d58847ddbc5168e872d6cf80)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:25:59.876142 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35bd69d-166f-461c-891b-711a59d152c5 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:25:59.878567 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 581/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:25:59 2022] GET /identity/v3/users/61d02255d58847ddbc5168e872d6cf80 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:25:59.883545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f67e8-4290-4688-a387-4f3a822b479f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:25:59.982401 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f67e8-4290-4688-a387-4f3a822b479f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:00.013588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6179c-99fa-46c5-98e7-e227cc494138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:00.014195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6179c-99fa-46c5-98e7-e227cc494138 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:00.014466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6179c-99fa-46c5-98e7-e227cc494138 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:00.014727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6179c-99fa-46c5-98e7-e227cc494138 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:00.059185 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:00.080313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f6179c-99fa-46c5-98e7-e227cc494138 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:00.081781 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f6179c-99fa-46c5-98e7-e227cc494138 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:00.178612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 578/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:25:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:26:00.185432 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bd69d-166f-461c-891b-711a59d152c5 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:00.282868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bd69d-166f-461c-891b-711a59d152c5 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:00.314757 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c1055-32c0-4395-9418-94c4a814c2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:00.315361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1055-32c0-4395-9418-94c4a814c2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:00.315634 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1055-32c0-4395-9418-94c4a814c2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:00.315900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1055-32c0-4395-9418-94c4a814c2d7 None admin] PATH_INFO: `/v3/projects/da2d23cd274c4b639a7a2aecdc4db1cb` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:00.316979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c1055-32c0-4395-9418-94c4a814c2d7 None admin] RBAC: Authorizing `identity:get_project(project_id=da2d23cd274c4b639a7a2aecdc4db1cb)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:00.318425 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c1055-32c0-4395-9418-94c4a814c2d7 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:00.319999 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 582/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:26:00 2022] GET /identity/v3/projects/da2d23cd274c4b639a7a2aecdc4db1cb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:00.326739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6179c-99fa-46c5-98e7-e227cc494138 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:00.400608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:00.421068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6179c-99fa-46c5-98e7-e227cc494138 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:00.453748 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbe125-ab23-421f-ab07-7c85f585b5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:00.454357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbe125-ab23-421f-ab07-7c85f585b5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:00.454626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbe125-ab23-421f-ab07-7c85f585b5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:00.454890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbe125-ab23-421f-ab07-7c85f585b5e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:00.455583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbe125-ab23-421f-ab07-7c85f585b5e6 None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=da2d23cd274c4b639a7a2aecdc4db1cb, user.id=61d02255d58847ddbc5168e872d6cf80) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:26:00.455898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbe125-ab23-421f-ab07-7c85f585b5e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:00.457263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbe125-ab23-421f-ab07-7c85f585b5e6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:00.462716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 579/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:26:00 2022] GET /identity/v3/role_assignments?user.id=61d02255d58847ddbc5168e872d6cf80&scope.project.id=da2d23cd274c4b639a7a2aecdc4db1cb&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:02.614975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93274b94-ca82-4c7f-b917-61809612422c None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:02.615524 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93274b94-ca82-4c7f-b917-61809612422c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:02.615524 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93274b94-ca82-4c7f-b917-61809612422c None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:02.616823 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 583/1162] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:26:02.622291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e0825-6011-4813-aa3d-faa2bbac5e67 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:02.622739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e0825-6011-4813-aa3d-faa2bbac5e67 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:02.622838 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e0825-6011-4813-aa3d-faa2bbac5e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:02.640994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-7c7e0825-6011-4813-aa3d-faa2bbac5e67 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:26:02.642430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7c7e0825-6011-4813-aa3d-faa2bbac5e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:26:02.705324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 580/1163] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:26:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:26:02.970718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93274b94-ca82-4c7f-b917-61809612422c None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:03.016601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:03.076496 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93274b94-ca82-4c7f-b917-61809612422c None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:03.109962 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d58d8-86d3-4b8f-b048-6f24d1cbe98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:03.110559 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d58d8-86d3-4b8f-b048-6f24d1cbe98e None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:03.110832 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d58d8-86d3-4b8f-b048-6f24d1cbe98e None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:03.111098 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d58d8-86d3-4b8f-b048-6f24d1cbe98e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:03.175572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d58d8-86d3-4b8f-b048-6f24d1cbe98e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:03.176929 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d58d8-86d3-4b8f-b048-6f24d1cbe98e None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:03.219834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:03.272503 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 584/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:26:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:26:03.278789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e0825-6011-4813-aa3d-faa2bbac5e67 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:03.345596 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:03.374210 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e0825-6011-4813-aa3d-faa2bbac5e67 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:03.407374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c347a-fafb-4183-9607-e3ecc12308a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:03.407980 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c347a-fafb-4183-9607-e3ecc12308a9 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:03.408251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c347a-fafb-4183-9607-e3ecc12308a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:03.408515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c347a-fafb-4183-9607-e3ecc12308a9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:03.414902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c347a-fafb-4183-9607-e3ecc12308a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:03.416307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c347a-fafb-4183-9607-e3ecc12308a9 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-441c347a-fafb-4183-9607-e3ecc12308a9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:26:03.428261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:26:03.429559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:26:03.430802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 581/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 16 05:26:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:26:03.435953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d58d8-86d3-4b8f-b048-6f24d1cbe98e None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:03.530237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d58d8-86d3-4b8f-b048-6f24d1cbe98e None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:03.563121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332995aa-f493-4684-88db-bf5004238d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:03.563707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332995aa-f493-4684-88db-bf5004238d7a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:03.563980 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332995aa-f493-4684-88db-bf5004238d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:03.564246 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332995aa-f493-4684-88db-bf5004238d7a None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:03.564872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332995aa-f493-4684-88db-bf5004238d7a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:26:03.565166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332995aa-f493-4684-88db-bf5004238d7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:03.566536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332995aa-f493-4684-88db-bf5004238d7a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:03.573320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 585/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Fri Sep 16 05:26:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:05.718416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589174b4-23ca-468a-aaa9-4e835e87e96c None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:05.718972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589174b4-23ca-468a-aaa9-4e835e87e96c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:05.718972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589174b4-23ca-468a-aaa9-4e835e87e96c None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:05.720268 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 582/1167] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:26:05.725415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489acc4b-4ee0-4c09-926d-60ea96cd6712 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:05.725794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489acc4b-4ee0-4c09-926d-60ea96cd6712 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:05.725990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489acc4b-4ee0-4c09-926d-60ea96cd6712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:05.744106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-489acc4b-4ee0-4c09-926d-60ea96cd6712 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:26:05.745282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-489acc4b-4ee0-4c09-926d-60ea96cd6712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:26:05.747914 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:05.810181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 586/1168] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:26:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:26:06.081087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589174b4-23ca-468a-aaa9-4e835e87e96c None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:06.188952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589174b4-23ca-468a-aaa9-4e835e87e96c None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:06.218232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:06.222878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a29e539-7d2e-404d-96ab-cf02de7839e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:06.223470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29e539-7d2e-404d-96ab-cf02de7839e9 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:06.223739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29e539-7d2e-404d-96ab-cf02de7839e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:06.224001 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29e539-7d2e-404d-96ab-cf02de7839e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:06.224691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29e539-7d2e-404d-96ab-cf02de7839e9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:06.226060 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a29e539-7d2e-404d-96ab-cf02de7839e9 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:06.305384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 583/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Fri Sep 16 05:26:06 2022] POST /identity/v3/domains => generated 267 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:26:08.438660 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00950f17-e7cf-4d55-8c1c-83daa336c5be None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:08.439284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00950f17-e7cf-4d55-8c1c-83daa336c5be None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:08.439284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00950f17-e7cf-4d55-8c1c-83daa336c5be None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:08.440514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 587/1170] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:26:08.445538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405af74-007f-461a-be2c-4f9998c54cb5 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:08.445985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405af74-007f-461a-be2c-4f9998c54cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:08.446090 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405af74-007f-461a-be2c-4f9998c54cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:08.464150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-4405af74-007f-461a-be2c-4f9998c54cb5 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:26:08.465345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4405af74-007f-461a-be2c-4f9998c54cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:26:08.529380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 584/1171] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:26:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:26:08.806444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00950f17-e7cf-4d55-8c1c-83daa336c5be None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:08.905089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00950f17-e7cf-4d55-8c1c-83daa336c5be None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:08.938455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d5fb3-4a76-4997-8f45-1e747783f60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:08.939081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d5fb3-4a76-4997-8f45-1e747783f60a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:08.939355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d5fb3-4a76-4997-8f45-1e747783f60a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:08.939623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d5fb3-4a76-4997-8f45-1e747783f60a None admin] PATH_INFO: `/v3/domains/d4d27db45cf54a409f2b8a878c64d7ad` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:08.945968 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:08.950696 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d5fb3-4a76-4997-8f45-1e747783f60a None admin] RBAC: Authorizing `identity:get_domain(domain_id=d4d27db45cf54a409f2b8a878c64d7ad)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:08.952122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d5fb3-4a76-4997-8f45-1e747783f60a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:08.953705 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 588/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Fri Sep 16 05:26:08 2022] GET /identity/v3/domains/d4d27db45cf54a409f2b8a878c64d7ad => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:08.959763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405af74-007f-461a-be2c-4f9998c54cb5 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:09.059892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405af74-007f-461a-be2c-4f9998c54cb5 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:09.083940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:09.095899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cc07b-aafa-43ff-80d5-9bc8983aa658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:09.097597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cc07b-aafa-43ff-80d5-9bc8983aa658 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:09.097597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cc07b-aafa-43ff-80d5-9bc8983aa658 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:09.097597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cc07b-aafa-43ff-80d5-9bc8983aa658 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:09.097896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cc07b-aafa-43ff-80d5-9bc8983aa658 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:09.099141 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cc07b-aafa-43ff-80d5-9bc8983aa658 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:09.182461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 585/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:26:08 2022] POST /identity/v3/projects => generated 361 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:26:11.345376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e29f0-f0bc-4aa4-a5fc-17aeefeb5289 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:11.345895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e29f0-f0bc-4aa4-a5fc-17aeefeb5289 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:11.345895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e29f0-f0bc-4aa4-a5fc-17aeefeb5289 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:11.347179 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 589/1174] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:26:11.352440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ea574-b004-4a8a-abc9-e2f6000039cb None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:11.352832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ea574-b004-4a8a-abc9-e2f6000039cb None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:11.352985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ea574-b004-4a8a-abc9-e2f6000039cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:11.370909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f70ea574-b004-4a8a-abc9-e2f6000039cb None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:26:11.372957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f70ea574-b004-4a8a-abc9-e2f6000039cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:26:11.438802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 586/1175] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:26:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:26:11.707363 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e29f0-f0bc-4aa4-a5fc-17aeefeb5289 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:11.805045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4e29f0-f0bc-4aa4-a5fc-17aeefeb5289 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:11.836202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80e5f1-cc8a-4005-b139-199bc35617be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:11.836802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80e5f1-cc8a-4005-b139-199bc35617be None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:11.837071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80e5f1-cc8a-4005-b139-199bc35617be None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:11.837334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80e5f1-cc8a-4005-b139-199bc35617be None admin] PATH_INFO: `/v3/domains/d4d27db45cf54a409f2b8a878c64d7ad` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:11.838413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80e5f1-cc8a-4005-b139-199bc35617be None admin] RBAC: Authorizing `identity:get_domain(domain_id=d4d27db45cf54a409f2b8a878c64d7ad)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:11.840368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80e5f1-cc8a-4005-b139-199bc35617be None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:11.841908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 590/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Fri Sep 16 05:26:11 2022] GET /identity/v3/domains/d4d27db45cf54a409f2b8a878c64d7ad => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:11.847643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ea574-b004-4a8a-abc9-e2f6000039cb None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:11.942404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ea574-b004-4a8a-abc9-e2f6000039cb None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:11.944618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:11.974840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf6046-aad7-42a0-ab66-4ba5aa900408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:11.975371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf6046-aad7-42a0-ab66-4ba5aa900408 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:11.975640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf6046-aad7-42a0-ab66-4ba5aa900408 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:11.975902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf6046-aad7-42a0-ab66-4ba5aa900408 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:11.976544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf6046-aad7-42a0-ab66-4ba5aa900408 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:11.977966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf6046-aad7-42a0-ab66-4ba5aa900408 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:12.060878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 587/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:26:11 2022] POST /identity/v3/users => generated 311 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:26:14.265664 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e66a1a-b039-460b-bb58-c45c85558831 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:14.266208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e66a1a-b039-460b-bb58-c45c85558831 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:14.266208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e66a1a-b039-460b-bb58-c45c85558831 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:14.267754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 591/1178] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:26:14.272944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4ab3a-6ee3-4001-9ffa-b8e42db17a3e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:14.273373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4ab3a-6ee3-4001-9ffa-b8e42db17a3e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:14.273472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4ab3a-6ee3-4001-9ffa-b8e42db17a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:14.293544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-95a4ab3a-6ee3-4001-9ffa-b8e42db17a3e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:26:14.295357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-95a4ab3a-6ee3-4001-9ffa-b8e42db17a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:26:14.359002 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 588/1179] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:26:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:26:14.630538 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e66a1a-b039-460b-bb58-c45c85558831 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:14.697482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:14.732854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e66a1a-b039-460b-bb58-c45c85558831 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:14.765888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb49c5-ed06-44a3-bce7-9ff94cdfea69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:14.766502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb49c5-ed06-44a3-bce7-9ff94cdfea69 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:14.767034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb49c5-ed06-44a3-bce7-9ff94cdfea69 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:14.767303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb49c5-ed06-44a3-bce7-9ff94cdfea69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:14.778543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb49c5-ed06-44a3-bce7-9ff94cdfea69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-b5bb49c5-ed06-44a3-bce7-9ff94cdfea69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:26:14.783333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:26:14.784477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:26:14.785543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:26:14.785543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:26:14.785543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:26:14.785543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:26:14.785543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:26:14.785543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:26:14.785543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:26:14.785543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:26:14.785543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:26:14.785543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:26:14.785543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:26:14.785543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:26:14.785543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 592/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:26:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:26:14.790497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4ab3a-6ee3-4001-9ffa-b8e42db17a3e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:14.878265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:14.889188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4ab3a-6ee3-4001-9ffa-b8e42db17a3e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:14.920844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e563da-ad8c-4fc4-9fb9-cf0bd2b9d1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:14.921931 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e563da-ad8c-4fc4-9fb9-cf0bd2b9d1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:14.921931 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e563da-ad8c-4fc4-9fb9-cf0bd2b9d1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:14.922091 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e563da-ad8c-4fc4-9fb9-cf0bd2b9d1b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:14.922660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e563da-ad8c-4fc4-9fb9-cf0bd2b9d1b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:26:14.922953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e563da-ad8c-4fc4-9fb9-cf0bd2b9d1b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:14.924358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e563da-ad8c-4fc4-9fb9-cf0bd2b9d1b6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:14.932719 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 589/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:26:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:26:14.938914 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb49c5-ed06-44a3-bce7-9ff94cdfea69 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:15.034234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb49c5-ed06-44a3-bce7-9ff94cdfea69 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:15.058953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:15.067215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a4747-0d30-4906-b53f-b37d71f7fbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:15.067828 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a4747-0d30-4906-b53f-b37d71f7fbbb None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:15.068103 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a4747-0d30-4906-b53f-b37d71f7fbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:15.068371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a4747-0d30-4906-b53f-b37d71f7fbbb None admin] PATH_INFO: `/v3/users/e363644e5d04424298eab06904a6afb7` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:15.086278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a4747-0d30-4906-b53f-b37d71f7fbbb None admin] RBAC: Authorizing `identity:get_user(user_id=e363644e5d04424298eab06904a6afb7)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:15.089019 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a4747-0d30-4906-b53f-b37d71f7fbbb None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:15.091807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 593/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:26:14 2022] GET /identity/v3/users/e363644e5d04424298eab06904a6afb7 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:15.097703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e563da-ad8c-4fc4-9fb9-cf0bd2b9d1b6 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:15.196144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e563da-ad8c-4fc4-9fb9-cf0bd2b9d1b6 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:15.228295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1f4c4-cb60-4673-b5b8-77d967600cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:15.228874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1f4c4-cb60-4673-b5b8-77d967600cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:15.229142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1f4c4-cb60-4673-b5b8-77d967600cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:15.229402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1f4c4-cb60-4673-b5b8-77d967600cc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:15.249453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:15.293736 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1f4c4-cb60-4673-b5b8-77d967600cc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:15.295174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1f4c4-cb60-4673-b5b8-77d967600cc2 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:15.389111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 590/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:26:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:26:15.395442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a4747-0d30-4906-b53f-b37d71f7fbbb None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:15.490289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a4747-0d30-4906-b53f-b37d71f7fbbb None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:15.523348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c186e-cc7d-47eb-b114-40a302bcdaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:15.523960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c186e-cc7d-47eb-b114-40a302bcdaf5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:15.524233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c186e-cc7d-47eb-b114-40a302bcdaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:15.524498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c186e-cc7d-47eb-b114-40a302bcdaf5 None admin] PATH_INFO: `/v3/projects/c1100b610e6a4cf1b6842ab0834af174` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:15.525614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c186e-cc7d-47eb-b114-40a302bcdaf5 None admin] RBAC: Authorizing `identity:get_project(project_id=c1100b610e6a4cf1b6842ab0834af174)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:15.527068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c186e-cc7d-47eb-b114-40a302bcdaf5 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:15.529519 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 594/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:26:15 2022] GET /identity/v3/projects/c1100b610e6a4cf1b6842ab0834af174 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:15.535598 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1f4c4-cb60-4673-b5b8-77d967600cc2 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:15.587543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:15.632911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1f4c4-cb60-4673-b5b8-77d967600cc2 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:15.665498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176b5da-0bf5-498e-a7fe-c20fa48020b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:15.666120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176b5da-0bf5-498e-a7fe-c20fa48020b2 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:15.666397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176b5da-0bf5-498e-a7fe-c20fa48020b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:15.666665 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176b5da-0bf5-498e-a7fe-c20fa48020b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:15.667376 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176b5da-0bf5-498e-a7fe-c20fa48020b2 None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=c1100b610e6a4cf1b6842ab0834af174, user.id=e363644e5d04424298eab06904a6afb7) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:26:15.667703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176b5da-0bf5-498e-a7fe-c20fa48020b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:15.669179 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176b5da-0bf5-498e-a7fe-c20fa48020b2 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:15.674562 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 591/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:26:15 2022] GET /identity/v3/role_assignments?user.id=e363644e5d04424298eab06904a6afb7&scope.project.id=c1100b610e6a4cf1b6842ab0834af174&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:17.861195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c55c7b-dd18-4de0-aa3b-9b4f7e6399cf None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:17.861758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c55c7b-dd18-4de0-aa3b-9b4f7e6399cf None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:17.861758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c55c7b-dd18-4de0-aa3b-9b4f7e6399cf None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:17.863091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 595/1186] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:26:17.868382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f82a9a-3744-45ea-8e22-21df178147de None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:17.868761 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f82a9a-3744-45ea-8e22-21df178147de None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:17.868936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f82a9a-3744-45ea-8e22-21df178147de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:17.887745 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-96f82a9a-3744-45ea-8e22-21df178147de None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:26:17.888897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-96f82a9a-3744-45ea-8e22-21df178147de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:26:17.953812 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 592/1187] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:26:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:26:18.223075 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c55c7b-dd18-4de0-aa3b-9b4f7e6399cf None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:18.275791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:18.324308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c55c7b-dd18-4de0-aa3b-9b4f7e6399cf None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:18.357514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80424fef-e150-49a2-a505-f8a4f068fd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:18.358170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80424fef-e150-49a2-a505-f8a4f068fd4c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:18.358446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80424fef-e150-49a2-a505-f8a4f068fd4c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:18.358712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80424fef-e150-49a2-a505-f8a4f068fd4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:18.368564 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80424fef-e150-49a2-a505-f8a4f068fd4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-80424fef-e150-49a2-a505-f8a4f068fd4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:26:18.374461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:26:18.375596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:26:18.376637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:26:18.376637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:26:18.376637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:26:18.376637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:26:18.376637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:26:18.376637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:26:18.376637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:26:18.376637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:26:18.376637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:26:18.376637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:26:18.376637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:26:18.376637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:26:18.376637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 596/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:26:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:26:18.381611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f82a9a-3744-45ea-8e22-21df178147de None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:18.426019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:18.475551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f82a9a-3744-45ea-8e22-21df178147de None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:18.508482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e8a8a-c8f1-48f6-9825-13c84166788e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:18.509068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e8a8a-c8f1-48f6-9825-13c84166788e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:18.509335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e8a8a-c8f1-48f6-9825-13c84166788e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:18.509593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e8a8a-c8f1-48f6-9825-13c84166788e None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:18.510264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e8a8a-c8f1-48f6-9825-13c84166788e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:26:18.510558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e8a8a-c8f1-48f6-9825-13c84166788e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:18.511941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40e8a8a-c8f1-48f6-9825-13c84166788e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:18.519771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 593/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:26:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:26:18.525495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80424fef-e150-49a2-a505-f8a4f068fd4c None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:18.619973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80424fef-e150-49a2-a505-f8a4f068fd4c None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:18.632393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:18.652684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec9692-a528-4212-bb98-f283354dcf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:18.653312 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec9692-a528-4212-bb98-f283354dcf3b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:18.653557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec9692-a528-4212-bb98-f283354dcf3b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:18.653836 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec9692-a528-4212-bb98-f283354dcf3b None admin] PATH_INFO: `/v3/users/e363644e5d04424298eab06904a6afb7` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:18.654937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec9692-a528-4212-bb98-f283354dcf3b None admin] RBAC: Authorizing `identity:get_user(user_id=e363644e5d04424298eab06904a6afb7)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:18.656334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec9692-a528-4212-bb98-f283354dcf3b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:18.657884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 597/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:26:18 2022] GET /identity/v3/users/e363644e5d04424298eab06904a6afb7 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:18.663674 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e8a8a-c8f1-48f6-9825-13c84166788e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:18.761335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e8a8a-c8f1-48f6-9825-13c84166788e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:18.770062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:18.794535 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2e163-6001-432e-970c-9299323f6af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:18.795148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2e163-6001-432e-970c-9299323f6af3 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:18.795417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2e163-6001-432e-970c-9299323f6af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:18.795679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2e163-6001-432e-970c-9299323f6af3 None admin] PATH_INFO: `/v3/projects/c1100b610e6a4cf1b6842ab0834af174` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:18.796766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2e163-6001-432e-970c-9299323f6af3 None admin] RBAC: Authorizing `identity:get_project(project_id=c1100b610e6a4cf1b6842ab0834af174)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:18.798507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2e163-6001-432e-970c-9299323f6af3 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:18.800608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 594/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:26:18 2022] GET /identity/v3/projects/c1100b610e6a4cf1b6842ab0834af174 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:18.806011 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec9692-a528-4212-bb98-f283354dcf3b None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:18.899378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec9692-a528-4212-bb98-f283354dcf3b None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:18.933985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64741cbd-c77e-4d70-bb5d-1efbbc0efb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:18.934625 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64741cbd-c77e-4d70-bb5d-1efbbc0efb1a None admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:18.934895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64741cbd-c77e-4d70-bb5d-1efbbc0efb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:18.935157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64741cbd-c77e-4d70-bb5d-1efbbc0efb1a None admin] PATH_INFO: `/v3/projects/c1100b610e6a4cf1b6842ab0834af174/users/e363644e5d04424298eab06904a6afb7/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:18.936996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64741cbd-c77e-4d70-bb5d-1efbbc0efb1a None admin] RBAC: Authorizing `identity:create_grant(project_id=c1100b610e6a4cf1b6842ab0834af174, user_id=e363644e5d04424298eab06904a6afb7, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:18.938439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64741cbd-c77e-4d70-bb5d-1efbbc0efb1a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:19.020553 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 598/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:26:18 2022] PUT /identity/v3/projects/c1100b610e6a4cf1b6842ab0834af174/users/e363644e5d04424298eab06904a6afb7/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:26:21.165008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ed374-28cd-455c-8468-7c28c1c3720a None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:21.165536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ed374-28cd-455c-8468-7c28c1c3720a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:21.165536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ed374-28cd-455c-8468-7c28c1c3720a None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:21.166894 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 595/1193] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:26:21.172884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bf8c0-ceb0-46ec-bfe9-f114676b7f65 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:21.173294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bf8c0-ceb0-46ec-bfe9-f114676b7f65 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:21.173388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0bf8c0-ceb0-46ec-bfe9-f114676b7f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:21.191232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-5e0bf8c0-ceb0-46ec-bfe9-f114676b7f65 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:26:21.192398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5e0bf8c0-ceb0-46ec-bfe9-f114676b7f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:26:21.225815 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:21.257064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 599/1194] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:26:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:26:21.537540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ed374-28cd-455c-8468-7c28c1c3720a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:21.637014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ed374-28cd-455c-8468-7c28c1c3720a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:21.669029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b847ff-ae53-4dab-b436-acad0cb7f8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:21.669628 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b847ff-ae53-4dab-b436-acad0cb7f8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:21.669925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b847ff-ae53-4dab-b436-acad0cb7f8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:21.670190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b847ff-ae53-4dab-b436-acad0cb7f8d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:21.680036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b847ff-ae53-4dab-b436-acad0cb7f8d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-41b847ff-ae53-4dab-b436-acad0cb7f8d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:26:21.685533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:26:21.686709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:26:21.687851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:26:21.687851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:26:21.687851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:26:21.687851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:26:21.687851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:26:21.687851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:26:21.687851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:26:21.687851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:26:21.687851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:26:21.687851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:26:21.687851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:26:21.687851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:26:21.687851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 596/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:26:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:26:21.692654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bf8c0-ceb0-46ec-bfe9-f114676b7f65 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:21.786443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0bf8c0-ceb0-46ec-bfe9-f114676b7f65 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:21.818302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247965a-867d-419f-b22f-d8905dd76305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:21.818828 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247965a-867d-419f-b22f-d8905dd76305 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:21.819096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247965a-867d-419f-b22f-d8905dd76305 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:21.819358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247965a-867d-419f-b22f-d8905dd76305 None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:21.819999 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247965a-867d-419f-b22f-d8905dd76305 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:26:21.820290 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247965a-867d-419f-b22f-d8905dd76305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:21.821627 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247965a-867d-419f-b22f-d8905dd76305 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:21.829729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 600/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:26:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:21.836289 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b847ff-ae53-4dab-b436-acad0cb7f8d2 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:21.871425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:21.933897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b847ff-ae53-4dab-b436-acad0cb7f8d2 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:21.967246 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f269d-0630-46e0-8d27-fd8cc57d2ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:21.967857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f269d-0630-46e0-8d27-fd8cc57d2ccd None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:21.968133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f269d-0630-46e0-8d27-fd8cc57d2ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:21.968404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f269d-0630-46e0-8d27-fd8cc57d2ccd None admin] PATH_INFO: `/v3/users/e363644e5d04424298eab06904a6afb7` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:21.969508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f269d-0630-46e0-8d27-fd8cc57d2ccd None admin] RBAC: Authorizing `identity:get_user(user_id=e363644e5d04424298eab06904a6afb7)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:21.971108 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f269d-0630-46e0-8d27-fd8cc57d2ccd None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:21.972657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 597/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Fri Sep 16 05:26:21 2022] GET /identity/v3/users/e363644e5d04424298eab06904a6afb7 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:21.978360 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247965a-867d-419f-b22f-d8905dd76305 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:22.008653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:22.075831 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247965a-867d-419f-b22f-d8905dd76305 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:22.108159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a035b-ba57-422e-9421-ac9f10960b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:22.108751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a035b-ba57-422e-9421-ac9f10960b0a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:22.109025 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a035b-ba57-422e-9421-ac9f10960b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:22.109293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a035b-ba57-422e-9421-ac9f10960b0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:22.174154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a035b-ba57-422e-9421-ac9f10960b0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:22.175574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a035b-ba57-422e-9421-ac9f10960b0a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:22.204916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:22.270774 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 601/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:26:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:26:22.276672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f269d-0630-46e0-8d27-fd8cc57d2ccd None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:22.373561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f269d-0630-46e0-8d27-fd8cc57d2ccd None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:22.375898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:22.406996 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd50294-c3d6-4785-ab4b-8b2671dbea9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:22.407607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd50294-c3d6-4785-ab4b-8b2671dbea9e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:22.407879 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd50294-c3d6-4785-ab4b-8b2671dbea9e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:22.408143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd50294-c3d6-4785-ab4b-8b2671dbea9e None admin] PATH_INFO: `/v3/projects/c1100b610e6a4cf1b6842ab0834af174` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:22.409239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd50294-c3d6-4785-ab4b-8b2671dbea9e None admin] RBAC: Authorizing `identity:get_project(project_id=c1100b610e6a4cf1b6842ab0834af174)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:22.410679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd50294-c3d6-4785-ab4b-8b2671dbea9e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:22.412254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 598/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Fri Sep 16 05:26:22 2022] GET /identity/v3/projects/c1100b610e6a4cf1b6842ab0834af174 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:22.418609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a035b-ba57-422e-9421-ac9f10960b0a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:26:22.513521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a035b-ba57-422e-9421-ac9f10960b0a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:26:22.541112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:22.545983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45710fed-45d8-44f9-a350-bead833dba64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:26:22.546568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45710fed-45d8-44f9-a350-bead833dba64 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:26:22.546839 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45710fed-45d8-44f9-a350-bead833dba64 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:26:22.547104 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45710fed-45d8-44f9-a350-bead833dba64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:26:22.547852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45710fed-45d8-44f9-a350-bead833dba64 None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=c1100b610e6a4cf1b6842ab0834af174, user.id=e363644e5d04424298eab06904a6afb7) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:26:22.548154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45710fed-45d8-44f9-a350-bead833dba64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:26:22.549558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45710fed-45d8-44f9-a350-bead833dba64 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:26:22.555254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 602/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:26:22 2022] GET /identity/v3/role_assignments?user.id=e363644e5d04424298eab06904a6afb7&scope.project.id=c1100b610e6a4cf1b6842ab0834af174&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:26:32.403676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=14 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:26:32.568897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:35.090616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884991-b6c7-4e4f-9f08-1635c1682f30 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:35.091194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884991-b6c7-4e4f-9f08-1635c1682f30 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:35.091194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1884991-b6c7-4e4f-9f08-1635c1682f30 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:35.092455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 599/1201] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:27:35.098654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bba8dd-c849-4992-a3f6-03cc8df5638c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:35.099079 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bba8dd-c849-4992-a3f6-03cc8df5638c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:35.099208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bba8dd-c849-4992-a3f6-03cc8df5638c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:35.118485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-21bba8dd-c849-4992-a3f6-03cc8df5638c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:27:35.119665 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-21bba8dd-c849-4992-a3f6-03cc8df5638c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:27:35.185193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 603/1202] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:27:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:27:35.461190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1884991-b6c7-4e4f-9f08-1635c1682f30 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:35.563555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1884991-b6c7-4e4f-9f08-1635c1682f30 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:35.596208 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481fb42-df9e-436a-ab77-f2f5e3bcfbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:35.596820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481fb42-df9e-436a-ab77-f2f5e3bcfbbd None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:35.597088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481fb42-df9e-436a-ab77-f2f5e3bcfbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:35.597349 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a481fb42-df9e-436a-ab77-f2f5e3bcfbbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:35.606963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481fb42-df9e-436a-ab77-f2f5e3bcfbbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:35.608407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a481fb42-df9e-436a-ab77-f2f5e3bcfbbd None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-a481fb42-df9e-436a-ab77-f2f5e3bcfbbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:27:35.622989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:35.624202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:27:35.626187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:27:35.626187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:35.626187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:35.626187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 600/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:27:35.630999 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bba8dd-c849-4992-a3f6-03cc8df5638c None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:35.729891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bba8dd-c849-4992-a3f6-03cc8df5638c None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:35.758202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:35.763837 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce20fef2-d668-494c-a606-3f90fbaff02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:35.764438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20fef2-d668-494c-a606-3f90fbaff02a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:35.764712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20fef2-d668-494c-a606-3f90fbaff02a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:35.764979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce20fef2-d668-494c-a606-3f90fbaff02a None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:35.770022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20fef2-d668-494c-a606-3f90fbaff02a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:35.770022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20fef2-d668-494c-a606-3f90fbaff02a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:35.770022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce20fef2-d668-494c-a606-3f90fbaff02a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:35.778993 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 604/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:35.785705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481fb42-df9e-436a-ab77-f2f5e3bcfbbd None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:35.838126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:35.885719 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a481fb42-df9e-436a-ab77-f2f5e3bcfbbd None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:35.919761 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d422835-890d-47c3-a4e8-c9fe2a839893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:35.920359 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d422835-890d-47c3-a4e8-c9fe2a839893 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:35.920631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d422835-890d-47c3-a4e8-c9fe2a839893 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:35.920936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d422835-890d-47c3-a4e8-c9fe2a839893 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:35.921531 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d422835-890d-47c3-a4e8-c9fe2a839893 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:35.923832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d422835-890d-47c3-a4e8-c9fe2a839893 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:36.024002 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 601/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Fri Sep 16 05:27:35 2022] POST /identity/v3/users => generated 251 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:27:38.231096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2378fd-89b5-42bc-a5b7-a0c8cdbb4dbc None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:38.231648 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2378fd-89b5-42bc-a5b7-a0c8cdbb4dbc None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:38.231648 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2378fd-89b5-42bc-a5b7-a0c8cdbb4dbc None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:38.232930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 605/1206] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:27:38.238946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54e062-8ccf-405e-8a72-29e9cbd21efa None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:38.239348 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54e062-8ccf-405e-8a72-29e9cbd21efa None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:38.239470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54e062-8ccf-405e-8a72-29e9cbd21efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:38.257254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-5e54e062-8ccf-405e-8a72-29e9cbd21efa None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:27:38.258446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5e54e062-8ccf-405e-8a72-29e9cbd21efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:27:38.324885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 602/1207] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:27:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:27:38.589954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2378fd-89b5-42bc-a5b7-a0c8cdbb4dbc None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:38.688605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2378fd-89b5-42bc-a5b7-a0c8cdbb4dbc None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:38.721136 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111b6ab-6c16-44d9-90dc-90e6675495b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:38.721743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111b6ab-6c16-44d9-90dc-90e6675495b5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:38.722038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111b6ab-6c16-44d9-90dc-90e6675495b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:38.722300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111b6ab-6c16-44d9-90dc-90e6675495b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:38.732426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111b6ab-6c16-44d9-90dc-90e6675495b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-d111b6ab-6c16-44d9-90dc-90e6675495b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:27:38.737937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:38.739147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:38.740031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:38.740031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:38.740031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:38.740031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:38.740031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:27:38.740031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:27:38.740031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:27:38.740031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:27:38.740031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:27:38.740031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:27:38.740031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:27:38.740031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:27:38.740031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 606/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:27:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:38.745103 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54e062-8ccf-405e-8a72-29e9cbd21efa None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:38.784103 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:38.844359 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54e062-8ccf-405e-8a72-29e9cbd21efa None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:38.877734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379bbdc-cf14-4ca7-9a3a-ba48f033e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:38.878357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379bbdc-cf14-4ca7-9a3a-ba48f033e13b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:38.878633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379bbdc-cf14-4ca7-9a3a-ba48f033e13b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:38.878900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379bbdc-cf14-4ca7-9a3a-ba48f033e13b None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:38.879551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379bbdc-cf14-4ca7-9a3a-ba48f033e13b None admin] RBAC: Adding query filter params (name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:38.879849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379bbdc-cf14-4ca7-9a3a-ba48f033e13b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:38.881289 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379bbdc-cf14-4ca7-9a3a-ba48f033e13b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:38.889524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 603/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:27:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:38.895839 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111b6ab-6c16-44d9-90dc-90e6675495b5 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:38.948842 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:38.997398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111b6ab-6c16-44d9-90dc-90e6675495b5 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:39.030812 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60948de-f15b-4fba-9d32-ccde4e712df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:39.032177 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60948de-f15b-4fba-9d32-ccde4e712df2 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:39.032554 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60948de-f15b-4fba-9d32-ccde4e712df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:39.032825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60948de-f15b-4fba-9d32-ccde4e712df2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:39.043098 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60948de-f15b-4fba-9d32-ccde4e712df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:39.044646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60948de-f15b-4fba-9d32-ccde4e712df2 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-f60948de-f15b-4fba-9d32-ccde4e712df2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:27:39.058719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:39.059902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:27:39.060834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:27:39.060834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:39.060834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:27:39.060834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 607/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:39.065883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379bbdc-cf14-4ca7-9a3a-ba48f033e13b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:39.166072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:39.168974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379bbdc-cf14-4ca7-9a3a-ba48f033e13b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:39.201595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5f38f-cc7b-4ec8-b635-2ff3d680b4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:39.202215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5f38f-cc7b-4ec8-b635-2ff3d680b4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:39.202486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5f38f-cc7b-4ec8-b635-2ff3d680b4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:39.202750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5f38f-cc7b-4ec8-b635-2ff3d680b4e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:39.204192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5f38f-cc7b-4ec8-b635-2ff3d680b4e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:39.204493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5f38f-cc7b-4ec8-b635-2ff3d680b4e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:39.205901 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5f38f-cc7b-4ec8-b635-2ff3d680b4e5 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:39.216780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 604/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:39.224960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60948de-f15b-4fba-9d32-ccde4e712df2 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:39.326679 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60948de-f15b-4fba-9d32-ccde4e712df2 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:39.328202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:39.360761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f9753-23de-447c-b1c4-96252db97dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:39.361519 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f9753-23de-447c-b1c4-96252db97dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:39.362121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f9753-23de-447c-b1c4-96252db97dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:39.362389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f9753-23de-447c-b1c4-96252db97dd7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:39.370934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f9753-23de-447c-b1c4-96252db97dd7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:39.372355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f9753-23de-447c-b1c4-96252db97dd7 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-013f9753-23de-447c-b1c4-96252db97dd7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:27:39.384017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:39.385198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:39.386256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:39.386256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:39.386256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:27:39.386256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:27:39.386256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:27:39.386256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:27:39.386256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:27:39.386256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:27:39.386256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 05:27:39.386256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:27:39.386256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 608/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:39.391498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5f38f-cc7b-4ec8-b635-2ff3d680b4e5 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:39.499852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5f38f-cc7b-4ec8-b635-2ff3d680b4e5 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:39.533155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426ca21-8fba-4003-ad76-4ed1212efcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:39.533745 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426ca21-8fba-4003-ad76-4ed1212efcf0 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:39.534044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426ca21-8fba-4003-ad76-4ed1212efcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:39.534312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426ca21-8fba-4003-ad76-4ed1212efcf0 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:39.535015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426ca21-8fba-4003-ad76-4ed1212efcf0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:39.535314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426ca21-8fba-4003-ad76-4ed1212efcf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:39.536744 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426ca21-8fba-4003-ad76-4ed1212efcf0 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:39.555190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 605/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Sep 16 05:27:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:39.561307 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f9753-23de-447c-b1c4-96252db97dd7 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:39.659687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f9753-23de-447c-b1c4-96252db97dd7 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:39.692440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9323c68-91da-4c8b-8262-46c28a6a1c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:39.693030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9323c68-91da-4c8b-8262-46c28a6a1c4d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:39.693301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9323c68-91da-4c8b-8262-46c28a6a1c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:39.693566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9323c68-91da-4c8b-8262-46c28a6a1c4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:39.723214 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:39.759314 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9323c68-91da-4c8b-8262-46c28a6a1c4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:39.760669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9323c68-91da-4c8b-8262-46c28a6a1c4d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:39.856388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 609/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:27:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:27:39.862999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426ca21-8fba-4003-ad76-4ed1212efcf0 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:39.882208 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:39.960941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426ca21-8fba-4003-ad76-4ed1212efcf0 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:39.997377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b472d-d379-42d6-b6dc-bb4d34dded3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:39.998021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b472d-d379-42d6-b6dc-bb4d34dded3c None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:39.998294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b472d-d379-42d6-b6dc-bb4d34dded3c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:39.998555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b472d-d379-42d6-b6dc-bb4d34dded3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:40.008741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b472d-d379-42d6-b6dc-bb4d34dded3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:40.016071 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b472d-d379-42d6-b6dc-bb4d34dded3c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-bb6b472d-d379-42d6-b6dc-bb4d34dded3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:27:40.027249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:40.028334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:27:40.029290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:27:40.029290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:40.029290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:40.029290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 606/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:27:40.034685 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9323c68-91da-4c8b-8262-46c28a6a1c4d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:40.094574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:40.130739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9323c68-91da-4c8b-8262-46c28a6a1c4d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:40.164160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80deb64-dd46-4623-a882-b6bfc9bc5e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:40.164769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80deb64-dd46-4623-a882-b6bfc9bc5e11 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:40.165062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80deb64-dd46-4623-a882-b6bfc9bc5e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:40.165331 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80deb64-dd46-4623-a882-b6bfc9bc5e11 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:40.165993 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80deb64-dd46-4623-a882-b6bfc9bc5e11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:40.166288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80deb64-dd46-4623-a882-b6bfc9bc5e11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:40.167669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80deb64-dd46-4623-a882-b6bfc9bc5e11 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:40.178482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 610/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:40.184166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b472d-d379-42d6-b6dc-bb4d34dded3c None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:40.252357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:40.282586 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b472d-d379-42d6-b6dc-bb4d34dded3c None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:40.315044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854b7a-599e-41b1-b234-fde928afa7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:40.315657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854b7a-599e-41b1-b234-fde928afa7cb None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:40.315932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854b7a-599e-41b1-b234-fde928afa7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:40.316195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9854b7a-599e-41b1-b234-fde928afa7cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:40.322622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9854b7a-599e-41b1-b234-fde928afa7cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:40.324042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9854b7a-599e-41b1-b234-fde928afa7cb None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-c9854b7a-599e-41b1-b234-fde928afa7cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:27:40.335634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:40.340270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:40.341404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:27:40.341404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:27:40.341404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:27:40.341404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:27:40.341404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:27:40.341404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:27:40.341404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:27:40.341404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:40.341404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 607/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:27:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:40.342791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80deb64-dd46-4623-a882-b6bfc9bc5e11 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:40.442178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80deb64-dd46-4623-a882-b6bfc9bc5e11 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:40.464240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:40.475420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc9739-b564-4125-85dc-4701a702177b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:40.476020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc9739-b564-4125-85dc-4701a702177b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:40.476294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc9739-b564-4125-85dc-4701a702177b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:40.476563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc9739-b564-4125-85dc-4701a702177b None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:40.477243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc9739-b564-4125-85dc-4701a702177b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:40.477544 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc9739-b564-4125-85dc-4701a702177b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:40.478990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc9739-b564-4125-85dc-4701a702177b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:40.490461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 611/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:27:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:27:40.496727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854b7a-599e-41b1-b234-fde928afa7cb None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:40.596028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9854b7a-599e-41b1-b234-fde928afa7cb None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:40.623113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:40.629382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468069e9-c3de-45ac-a64f-a263a6dbbe88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:40.630003 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468069e9-c3de-45ac-a64f-a263a6dbbe88 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:40.630271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468069e9-c3de-45ac-a64f-a263a6dbbe88 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:40.630547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468069e9-c3de-45ac-a64f-a263a6dbbe88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:40.631232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468069e9-c3de-45ac-a64f-a263a6dbbe88 None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=9975c0dab76f4ccbb4d2107e8e780ae5) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:40.631527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468069e9-c3de-45ac-a64f-a263a6dbbe88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:40.632922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468069e9-c3de-45ac-a64f-a263a6dbbe88 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:40.638200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 608/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:27:40 2022] GET /identity/v3/role_assignments?user.id=9975c0dab76f4ccbb4d2107e8e780ae5&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:42.822329 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4cfeab-8982-4af1-836b-0d14cb398ec8 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:42.822880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4cfeab-8982-4af1-836b-0d14cb398ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:42.822880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4cfeab-8982-4af1-836b-0d14cb398ec8 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:42.824193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 612/1220] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:27:42.829706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3a05a-f63e-484b-ab8d-02898d19eaea None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:42.830232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3a05a-f63e-484b-ab8d-02898d19eaea None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:42.830330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3a05a-f63e-484b-ab8d-02898d19eaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:42.848191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-e7b3a05a-f63e-484b-ab8d-02898d19eaea None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:27:42.849363 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e7b3a05a-f63e-484b-ab8d-02898d19eaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:27:42.914602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 609/1221] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:27:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:27:43.204330 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4cfeab-8982-4af1-836b-0d14cb398ec8 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:43.305897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4cfeab-8982-4af1-836b-0d14cb398ec8 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:43.339894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb7aef-9fda-4a6a-a44d-108b6fe11bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:43.340523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb7aef-9fda-4a6a-a44d-108b6fe11bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:43.340796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb7aef-9fda-4a6a-a44d-108b6fe11bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:43.341063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb7aef-9fda-4a6a-a44d-108b6fe11bb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:43.351755 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb7aef-9fda-4a6a-a44d-108b6fe11bb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-babb7aef-9fda-4a6a-a44d-108b6fe11bb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:27:43.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:43.357808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:43.364821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:43.364821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:43.364821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:43.364821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:43.364821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:27:43.364821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:27:43.364821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:27:43.364821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:27:43.364821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:27:43.364821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:27:43.364821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:27:43.364821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:27:43.364821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 613/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:27:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:43.365702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3a05a-f63e-484b-ab8d-02898d19eaea None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:43.465055 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3a05a-f63e-484b-ab8d-02898d19eaea None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:43.494318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:43.504726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568aa36-607f-4769-ab87-4332e450dc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:43.505323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568aa36-607f-4769-ab87-4332e450dc12 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:43.505592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568aa36-607f-4769-ab87-4332e450dc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:43.505878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568aa36-607f-4769-ab87-4332e450dc12 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:43.506537 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568aa36-607f-4769-ab87-4332e450dc12 None admin] RBAC: Adding query filter params (name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:43.506833 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568aa36-607f-4769-ab87-4332e450dc12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:43.508212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568aa36-607f-4769-ab87-4332e450dc12 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:43.516529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 610/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:27:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:43.522573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb7aef-9fda-4a6a-a44d-108b6fe11bb9 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:43.565808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:43.621907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb7aef-9fda-4a6a-a44d-108b6fe11bb9 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:43.655308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af57f3-140e-411c-9359-287773a2603a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:43.655937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af57f3-140e-411c-9359-287773a2603a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:43.657187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af57f3-140e-411c-9359-287773a2603a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:43.657187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af57f3-140e-411c-9359-287773a2603a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:43.667558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af57f3-140e-411c-9359-287773a2603a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:43.668992 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af57f3-140e-411c-9359-287773a2603a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-f8af57f3-140e-411c-9359-287773a2603a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:27:43.683215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:43.684335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:27:43.685403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:27:43.685403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:43.685403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:27:43.685403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 614/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:43.691015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568aa36-607f-4769-ab87-4332e450dc12 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:43.790154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568aa36-607f-4769-ab87-4332e450dc12 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:43.823291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70d0bd-1f84-42b2-bb76-c7a7c94c82ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:43.823883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70d0bd-1f84-42b2-bb76-c7a7c94c82ba None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:43.824158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70d0bd-1f84-42b2-bb76-c7a7c94c82ba None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:43.824420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70d0bd-1f84-42b2-bb76-c7a7c94c82ba None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:43.825047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70d0bd-1f84-42b2-bb76-c7a7c94c82ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:43.825344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70d0bd-1f84-42b2-bb76-c7a7c94c82ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:43.826710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70d0bd-1f84-42b2-bb76-c7a7c94c82ba None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:43.838059 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 611/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:43.844028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af57f3-140e-411c-9359-287773a2603a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:43.937055 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:43.943788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af57f3-140e-411c-9359-287773a2603a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:43.976639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de656211-6167-4981-9ad7-a5dac4e8dd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:43.977252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de656211-6167-4981-9ad7-a5dac4e8dd6f None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:43.977521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de656211-6167-4981-9ad7-a5dac4e8dd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:43.977788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de656211-6167-4981-9ad7-a5dac4e8dd6f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:43.985661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de656211-6167-4981-9ad7-a5dac4e8dd6f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:43.987095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de656211-6167-4981-9ad7-a5dac4e8dd6f None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-de656211-6167-4981-9ad7-a5dac4e8dd6f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:27:43.998271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:43.999472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:44.000701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:44.000701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:44.000701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:27:44.000701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:27:44.000701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:27:44.000701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:27:44.000701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:27:44.000701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:27:44.000701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 05:27:44.000701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:27:44.000701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 615/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:44.005423 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70d0bd-1f84-42b2-bb76-c7a7c94c82ba None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:44.052319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:44.103028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70d0bd-1f84-42b2-bb76-c7a7c94c82ba None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:44.136371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6f60b-054e-402e-ac1b-d294a9c5b801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:44.137158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6f60b-054e-402e-ac1b-d294a9c5b801 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:44.137432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6f60b-054e-402e-ac1b-d294a9c5b801 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:44.137713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6f60b-054e-402e-ac1b-d294a9c5b801 None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:44.138456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6f60b-054e-402e-ac1b-d294a9c5b801 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:44.138755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6f60b-054e-402e-ac1b-d294a9c5b801 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:44.140154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6f60b-054e-402e-ac1b-d294a9c5b801 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:44.158088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 612/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Sep 16 05:27:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:27:44.164094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de656211-6167-4981-9ad7-a5dac4e8dd6f None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:44.261498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de656211-6167-4981-9ad7-a5dac4e8dd6f None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:44.295085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fe0c3-8aa4-4ec1-bb34-818bef58d58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:44.295690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fe0c3-8aa4-4ec1-bb34-818bef58d58d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:44.295960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fe0c3-8aa4-4ec1-bb34-818bef58d58d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:44.296254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fe0c3-8aa4-4ec1-bb34-818bef58d58d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:44.307524 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fe0c3-8aa4-4ec1-bb34-818bef58d58d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:44.308916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45fe0c3-8aa4-4ec1-bb34-818bef58d58d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-d45fe0c3-8aa4-4ec1-bb34-818bef58d58d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:27:44.322884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:44.324049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:27:44.325151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:27:44.325151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:44.325151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:27:44.325151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 616/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:44.330449 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6f60b-054e-402e-ac1b-d294a9c5b801 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:44.433763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6f60b-054e-402e-ac1b-d294a9c5b801 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:44.436042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:44.466905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415f1f7-3aca-4db1-835e-39944f407ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:44.467499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415f1f7-3aca-4db1-835e-39944f407ba6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:44.467774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415f1f7-3aca-4db1-835e-39944f407ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:44.468038 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415f1f7-3aca-4db1-835e-39944f407ba6 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:44.469262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415f1f7-3aca-4db1-835e-39944f407ba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:44.469640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415f1f7-3aca-4db1-835e-39944f407ba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:44.471030 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415f1f7-3aca-4db1-835e-39944f407ba6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:44.481700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 613/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:44.487807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fe0c3-8aa4-4ec1-bb34-818bef58d58d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:44.496911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:44.585325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fe0c3-8aa4-4ec1-bb34-818bef58d58d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:44.618431 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f4337-e2f3-4f28-bbb6-2e93c4cb005d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:44.619047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f4337-e2f3-4f28-bbb6-2e93c4cb005d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:44.619316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f4337-e2f3-4f28-bbb6-2e93c4cb005d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:44.619579 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f4337-e2f3-4f28-bbb6-2e93c4cb005d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:44.626332 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f4337-e2f3-4f28-bbb6-2e93c4cb005d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:44.627746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f4337-e2f3-4f28-bbb6-2e93c4cb005d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-ef9f4337-e2f3-4f28-bbb6-2e93c4cb005d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:27:44.639517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:44.641038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:44.642368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:27:44.642368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:27:44.642368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:27:44.642368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:27:44.642368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:27:44.642368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:27:44.642368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:27:44.642368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:27:44.642368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 617/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:27:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:44.647906 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415f1f7-3aca-4db1-835e-39944f407ba6 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:44.748286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415f1f7-3aca-4db1-835e-39944f407ba6 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:44.782732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e7c0f-4b1e-4ed5-8d25-0756a854ba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:44.783327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e7c0f-4b1e-4ed5-8d25-0756a854ba01 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:44.783597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e7c0f-4b1e-4ed5-8d25-0756a854ba01 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:44.783855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e7c0f-4b1e-4ed5-8d25-0756a854ba01 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:44.784521 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e7c0f-4b1e-4ed5-8d25-0756a854ba01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:44.784813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e7c0f-4b1e-4ed5-8d25-0756a854ba01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:44.786254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e7c0f-4b1e-4ed5-8d25-0756a854ba01 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:44.797862 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 614/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:27:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:44.803975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f4337-e2f3-4f28-bbb6-2e93c4cb005d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:44.870467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:44.903804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f4337-e2f3-4f28-bbb6-2e93c4cb005d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:44.939091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c1764-af3a-4c29-8947-9c84fa506822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:44.939741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c1764-af3a-4c29-8947-9c84fa506822 None admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:44.940013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c1764-af3a-4c29-8947-9c84fa506822 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:44.940279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c1764-af3a-4c29-8947-9c84fa506822 None admin] PATH_INFO: `/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/9975c0dab76f4ccbb4d2107e8e780ae5/roles/ee551384189f433bba7e06f47a794cf0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:44.955036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c1764-af3a-4c29-8947-9c84fa506822 None admin] RBAC: Authorizing `identity:create_grant(project_id=29b2e27932984de79ee8dfbe95312ac2, user_id=9975c0dab76f4ccbb4d2107e8e780ae5, role_id=ee551384189f433bba7e06f47a794cf0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:44.956452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c1764-af3a-4c29-8947-9c84fa506822 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:45.056680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 618/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:27:44 2022] PUT /identity/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/9975c0dab76f4ccbb4d2107e8e780ae5/roles/ee551384189f433bba7e06f47a794cf0 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:27:47.206196 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2d888-aeda-4116-a641-d5b135b0040b None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:47.206734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2d888-aeda-4116-a641-d5b135b0040b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:47.206734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2d888-aeda-4116-a641-d5b135b0040b None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:47.207992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 615/1233] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:27:47.213164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae41f6e-9eac-49be-8746-0426ebd7daf5 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:47.213564 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae41f6e-9eac-49be-8746-0426ebd7daf5 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:47.213700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae41f6e-9eac-49be-8746-0426ebd7daf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:47.231927 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-bae41f6e-9eac-49be-8746-0426ebd7daf5 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:27:47.233401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-bae41f6e-9eac-49be-8746-0426ebd7daf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:27:47.299941 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 619/1234] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:27:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:27:47.583890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2d888-aeda-4116-a641-d5b135b0040b None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:47.608989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:47.683126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2d888-aeda-4116-a641-d5b135b0040b None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:47.716995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69960c3-9aa1-4805-a5f5-3a95fd7170bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:47.717605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69960c3-9aa1-4805-a5f5-3a95fd7170bd None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:47.717900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69960c3-9aa1-4805-a5f5-3a95fd7170bd None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:47.718166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69960c3-9aa1-4805-a5f5-3a95fd7170bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:47.728246 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69960c3-9aa1-4805-a5f5-3a95fd7170bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-f69960c3-9aa1-4805-a5f5-3a95fd7170bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:27:47.732961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:47.734190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:47.735722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:47.735722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:47.735722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:47.735722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:47.735722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:27:47.735722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:27:47.735722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:27:47.735722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:27:47.735722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:27:47.735722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:27:47.735722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 16 05:27:47.735722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:47.735722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 616/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Fri Sep 16 05:27:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:47.740479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae41f6e-9eac-49be-8746-0426ebd7daf5 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:47.786851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:47.837251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae41f6e-9eac-49be-8746-0426ebd7daf5 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:47.869211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e4b38-6b64-4271-a3ef-69fbe601f22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:47.869812 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e4b38-6b64-4271-a3ef-69fbe601f22d None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:47.870115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e4b38-6b64-4271-a3ef-69fbe601f22d None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:47.870384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e4b38-6b64-4271-a3ef-69fbe601f22d None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:47.871042 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e4b38-6b64-4271-a3ef-69fbe601f22d None admin] RBAC: Adding query filter params (name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:47.871340 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e4b38-6b64-4271-a3ef-69fbe601f22d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:47.872696 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e4b38-6b64-4271-a3ef-69fbe601f22d None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:47.880820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 620/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Fri Sep 16 05:27:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:47.886755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69960c3-9aa1-4805-a5f5-3a95fd7170bd None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:47.969361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:47.984029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69960c3-9aa1-4805-a5f5-3a95fd7170bd None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:48.021280 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb5f76-b3a6-4cbf-af2a-aa90134aac1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:48.021926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb5f76-b3a6-4cbf-af2a-aa90134aac1a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:48.022194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb5f76-b3a6-4cbf-af2a-aa90134aac1a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:48.022455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb5f76-b3a6-4cbf-af2a-aa90134aac1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:48.032688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb5f76-b3a6-4cbf-af2a-aa90134aac1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:48.034124 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb5f76-b3a6-4cbf-af2a-aa90134aac1a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-0abb5f76-b3a6-4cbf-af2a-aa90134aac1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:27:48.048070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:48.052299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:27:48.053247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:27:48.053247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:48.053247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:48.053247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 617/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:48.055260 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e4b38-6b64-4271-a3ef-69fbe601f22d None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:48.153305 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e4b38-6b64-4271-a3ef-69fbe601f22d None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:48.162630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:48.186103 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c5b3f-973d-41fd-a438-cba09c1fbafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:48.186699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c5b3f-973d-41fd-a438-cba09c1fbafe None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:48.186974 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c5b3f-973d-41fd-a438-cba09c1fbafe None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:48.187360 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c5b3f-973d-41fd-a438-cba09c1fbafe None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:48.187900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c5b3f-973d-41fd-a438-cba09c1fbafe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:48.188195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c5b3f-973d-41fd-a438-cba09c1fbafe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:48.189538 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c5b3f-973d-41fd-a438-cba09c1fbafe None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:48.200782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 621/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:48.207448 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb5f76-b3a6-4cbf-af2a-aa90134aac1a None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:48.305343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb5f76-b3a6-4cbf-af2a-aa90134aac1a None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:48.341046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea0556-86d1-4d97-8512-a6cf829ea365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:48.342144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea0556-86d1-4d97-8512-a6cf829ea365 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:48.342605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea0556-86d1-4d97-8512-a6cf829ea365 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:48.343070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea0556-86d1-4d97-8512-a6cf829ea365 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:48.348129 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:48.353528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea0556-86d1-4d97-8512-a6cf829ea365 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:48.354988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea0556-86d1-4d97-8512-a6cf829ea365 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-81ea0556-86d1-4d97-8512-a6cf829ea365 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:27:48.366552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:48.367701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:48.369088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:48.369088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:48.369088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:27:48.369088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:27:48.369088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:27:48.369088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:27:48.369088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:27:48.369088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:27:48.369088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 05:27:48.369088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:48.369088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 618/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:48.373465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c5b3f-973d-41fd-a438-cba09c1fbafe None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:48.472570 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c5b3f-973d-41fd-a438-cba09c1fbafe None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:48.507091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01271809-656a-4c68-a47b-3c53beacfbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:48.507682 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01271809-656a-4c68-a47b-3c53beacfbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:48.507955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01271809-656a-4c68-a47b-3c53beacfbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:48.508267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01271809-656a-4c68-a47b-3c53beacfbf1 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:48.508974 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01271809-656a-4c68-a47b-3c53beacfbf1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:48.509271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01271809-656a-4c68-a47b-3c53beacfbf1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:48.510697 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01271809-656a-4c68-a47b-3c53beacfbf1 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:48.527947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 622/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Sep 16 05:27:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:48.534251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea0556-86d1-4d97-8512-a6cf829ea365 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:48.634105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea0556-86d1-4d97-8512-a6cf829ea365 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:48.667769 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3746c6f-ef4a-477d-9c25-24c8d1ac5b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:48.668354 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3746c6f-ef4a-477d-9c25-24c8d1ac5b9f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:48.668622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3746c6f-ef4a-477d-9c25-24c8d1ac5b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:48.668881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3746c6f-ef4a-477d-9c25-24c8d1ac5b9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:48.729447 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:48.736469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3746c6f-ef4a-477d-9c25-24c8d1ac5b9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:48.738101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3746c6f-ef4a-477d-9c25-24c8d1ac5b9f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:48.834204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 619/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:27:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:27:48.840142 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01271809-656a-4c68-a47b-3c53beacfbf1 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:48.866857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:48.937372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01271809-656a-4c68-a47b-3c53beacfbf1 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:48.971561 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad13fa7-f4f3-4a85-bac0-c24efb77b5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:48.972178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad13fa7-f4f3-4a85-bac0-c24efb77b5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:48.972452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad13fa7-f4f3-4a85-bac0-c24efb77b5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:48.972716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad13fa7-f4f3-4a85-bac0-c24efb77b5a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:48.982461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad13fa7-f4f3-4a85-bac0-c24efb77b5a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:48.983842 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad13fa7-f4f3-4a85-bac0-c24efb77b5a8 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-dad13fa7-f4f3-4a85-bac0-c24efb77b5a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:27:48.998061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:49.004207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:27:49.010489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:27:49.010489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:49.010489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:27:49.010489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 623/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:49.010777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3746c6f-ef4a-477d-9c25-24c8d1ac5b9f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:49.098452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:49.104103 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3746c6f-ef4a-477d-9c25-24c8d1ac5b9f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:49.138655 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0799f68-4b4d-4da7-bfd4-a2a7f58bfcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:49.139238 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0799f68-4b4d-4da7-bfd4-a2a7f58bfcc8 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:49.139505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0799f68-4b4d-4da7-bfd4-a2a7f58bfcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:49.139770 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0799f68-4b4d-4da7-bfd4-a2a7f58bfcc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:49.140390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0799f68-4b4d-4da7-bfd4-a2a7f58bfcc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:49.140680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0799f68-4b4d-4da7-bfd4-a2a7f58bfcc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:49.142674 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0799f68-4b4d-4da7-bfd4-a2a7f58bfcc8 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:49.153814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 620/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:49.160123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad13fa7-f4f3-4a85-bac0-c24efb77b5a8 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:49.237810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:49.258881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad13fa7-f4f3-4a85-bac0-c24efb77b5a8 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:49.291782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d123b1d-0874-4457-99e4-80f36b666cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:49.292403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d123b1d-0874-4457-99e4-80f36b666cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:49.292685 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d123b1d-0874-4457-99e4-80f36b666cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:49.292952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d123b1d-0874-4457-99e4-80f36b666cd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:49.300603 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d123b1d-0874-4457-99e4-80f36b666cd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:49.302064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d123b1d-0874-4457-99e4-80f36b666cd8 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-2d123b1d-0874-4457-99e4-80f36b666cd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:27:49.312973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:49.314219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:49.315442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:27:49.315442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:27:49.315442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:27:49.315442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:27:49.315442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:27:49.315442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:27:49.315442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:27:49.315442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:27:49.315442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 624/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:27:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:49.320520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0799f68-4b4d-4da7-bfd4-a2a7f58bfcc8 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:49.420477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0799f68-4b4d-4da7-bfd4-a2a7f58bfcc8 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:49.459842 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec468d7-91d7-4133-819f-197ed3dd3363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:49.460445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec468d7-91d7-4133-819f-197ed3dd3363 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:49.460722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec468d7-91d7-4133-819f-197ed3dd3363 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:49.460987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec468d7-91d7-4133-819f-197ed3dd3363 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:49.461802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec468d7-91d7-4133-819f-197ed3dd3363 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:49.462136 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec468d7-91d7-4133-819f-197ed3dd3363 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:49.463543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec468d7-91d7-4133-819f-197ed3dd3363 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:49.475367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 621/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:27:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:49.481734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d123b1d-0874-4457-99e4-80f36b666cd8 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:49.581311 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d123b1d-0874-4457-99e4-80f36b666cd8 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:49.614516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354c2b7-be80-4805-8480-ced0bd1b9a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:49.615103 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354c2b7-be80-4805-8480-ced0bd1b9a56 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:49.615373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354c2b7-be80-4805-8480-ced0bd1b9a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:49.615632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354c2b7-be80-4805-8480-ced0bd1b9a56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:49.616374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354c2b7-be80-4805-8480-ced0bd1b9a56 None admin] RBAC: Adding query filter params (role.id=ee551384189f433bba7e06f47a794cf0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=9975c0dab76f4ccbb4d2107e8e780ae5) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:49.616670 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354c2b7-be80-4805-8480-ced0bd1b9a56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:49.618088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354c2b7-be80-4805-8480-ced0bd1b9a56 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:49.624157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 625/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:27:49 2022] GET /identity/v3/role_assignments?user.id=9975c0dab76f4ccbb4d2107e8e780ae5&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=ee551384189f433bba7e06f47a794cf0 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:51.853111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639d69a-5293-42d0-9c78-4365ed9dd085 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:51.853700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639d69a-5293-42d0-9c78-4365ed9dd085 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:51.853700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639d69a-5293-42d0-9c78-4365ed9dd085 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:51.854980 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 622/1247] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:27:51.861080 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98532965-11d4-4e93-ac72-b4f89fefffd6 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:51.862545 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98532965-11d4-4e93-ac72-b4f89fefffd6 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:51.862545 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98532965-11d4-4e93-ac72-b4f89fefffd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:51.867814 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:51.880420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-98532965-11d4-4e93-ac72-b4f89fefffd6 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:27:51.881591 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-98532965-11d4-4e93-ac72-b4f89fefffd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:27:51.947054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 626/1248] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:27:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:27:52.218480 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639d69a-5293-42d0-9c78-4365ed9dd085 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:52.237575 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:52.322075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639d69a-5293-42d0-9c78-4365ed9dd085 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:52.356317 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03239794-7a63-4f46-a95c-a7772bab6571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:52.356928 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03239794-7a63-4f46-a95c-a7772bab6571 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:52.357200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03239794-7a63-4f46-a95c-a7772bab6571 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:52.357464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03239794-7a63-4f46-a95c-a7772bab6571 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:52.367799 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03239794-7a63-4f46-a95c-a7772bab6571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-03239794-7a63-4f46-a95c-a7772bab6571 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:27:52.372558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:52.374814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:52.376343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:52.376343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:52.376343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:52.376343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:52.376343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:27:52.376343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:27:52.376343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:27:52.376343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:27:52.376343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:27:52.376343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:27:52.376343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:27:52.376343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:52.376343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 623/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:27:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:52.380580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98532965-11d4-4e93-ac72-b4f89fefffd6 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:52.483642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98532965-11d4-4e93-ac72-b4f89fefffd6 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:52.513647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:52.518803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251593d-1ef5-4d34-b15c-a7d88febfafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:52.519451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251593d-1ef5-4d34-b15c-a7d88febfafc None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:52.519723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251593d-1ef5-4d34-b15c-a7d88febfafc None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:52.519988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251593d-1ef5-4d34-b15c-a7d88febfafc None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:52.520639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251593d-1ef5-4d34-b15c-a7d88febfafc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:52.520932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251593d-1ef5-4d34-b15c-a7d88febfafc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:52.522313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251593d-1ef5-4d34-b15c-a7d88febfafc None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:52.529968 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 627/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:27:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:52.536088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03239794-7a63-4f46-a95c-a7772bab6571 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:52.628701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:52.656637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03239794-7a63-4f46-a95c-a7772bab6571 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:52.690252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7400536-889c-4e48-90d8-459d77f3146a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:52.690866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7400536-889c-4e48-90d8-459d77f3146a None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:52.691135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7400536-889c-4e48-90d8-459d77f3146a None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:52.691399 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7400536-889c-4e48-90d8-459d77f3146a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:52.701041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7400536-889c-4e48-90d8-459d77f3146a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:52.702526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7400536-889c-4e48-90d8-459d77f3146a None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-b7400536-889c-4e48-90d8-459d77f3146a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:27:52.716965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:52.718106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:27:52.719955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:27:52.719955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:52.719955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:52.719955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 624/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:52.724550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251593d-1ef5-4d34-b15c-a7d88febfafc None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:52.822787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251593d-1ef5-4d34-b15c-a7d88febfafc None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:52.858389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14ad9c-d6f6-4a69-8b24-67a8181eb465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:52.858982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14ad9c-d6f6-4a69-8b24-67a8181eb465 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:52.859250 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14ad9c-d6f6-4a69-8b24-67a8181eb465 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:52.859510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14ad9c-d6f6-4a69-8b24-67a8181eb465 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:52.860667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14ad9c-d6f6-4a69-8b24-67a8181eb465 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:52.861132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14ad9c-d6f6-4a69-8b24-67a8181eb465 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:52.862619 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14ad9c-d6f6-4a69-8b24-67a8181eb465 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:52.874453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 628/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:52.880939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7400536-889c-4e48-90d8-459d77f3146a None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:52.981087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7400536-889c-4e48-90d8-459d77f3146a None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:53.006709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:53.014368 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5af9a2-54f0-492f-9bbb-3d9f4d685dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:53.015116 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5af9a2-54f0-492f-9bbb-3d9f4d685dbb None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:53.015415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5af9a2-54f0-492f-9bbb-3d9f4d685dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:53.015684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5af9a2-54f0-492f-9bbb-3d9f4d685dbb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:53.024115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5af9a2-54f0-492f-9bbb-3d9f4d685dbb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:53.025538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5af9a2-54f0-492f-9bbb-3d9f4d685dbb None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-7d5af9a2-54f0-492f-9bbb-3d9f4d685dbb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:27:53.037127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:53.038452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:53.039706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:53.039706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:53.039706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:27:53.039706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:27:53.039706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:27:53.039706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:27:53.039706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:27:53.039706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:27:53.039706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 05:27:53.039706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:53.039706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 625/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:27:53.044240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14ad9c-d6f6-4a69-8b24-67a8181eb465 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:53.101742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:53.145258 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14ad9c-d6f6-4a69-8b24-67a8181eb465 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:53.178955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bfd76-9433-445f-940d-fe6c598cc5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:53.179554 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bfd76-9433-445f-940d-fe6c598cc5dc None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:53.179830 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bfd76-9433-445f-940d-fe6c598cc5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:53.180100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575bfd76-9433-445f-940d-fe6c598cc5dc None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:53.180814 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575bfd76-9433-445f-940d-fe6c598cc5dc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:53.181113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575bfd76-9433-445f-940d-fe6c598cc5dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:53.182558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575bfd76-9433-445f-940d-fe6c598cc5dc None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:53.199046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 629/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Sep 16 05:27:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:53.204915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5af9a2-54f0-492f-9bbb-3d9f4d685dbb None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:53.306352 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5af9a2-54f0-492f-9bbb-3d9f4d685dbb None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:53.339339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57b1a4-ba7c-4825-b253-1fa1ad04573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:53.339937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57b1a4-ba7c-4825-b253-1fa1ad04573c None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:53.340212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57b1a4-ba7c-4825-b253-1fa1ad04573c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:53.340486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57b1a4-ba7c-4825-b253-1fa1ad04573c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:53.395165 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:53.406854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57b1a4-ba7c-4825-b253-1fa1ad04573c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:53.408298 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57b1a4-ba7c-4825-b253-1fa1ad04573c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:53.512666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 626/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:27:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 16 05:27:53.518796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bfd76-9433-445f-940d-fe6c598cc5dc None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:53.619143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575bfd76-9433-445f-940d-fe6c598cc5dc None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:53.632404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:53.652966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a95cb5-df6a-4eaf-982c-4b0305c3420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:53.653522 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a95cb5-df6a-4eaf-982c-4b0305c3420c None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:53.653789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a95cb5-df6a-4eaf-982c-4b0305c3420c None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:53.654085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a95cb5-df6a-4eaf-982c-4b0305c3420c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:53.664236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a95cb5-df6a-4eaf-982c-4b0305c3420c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:53.665647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a95cb5-df6a-4eaf-982c-4b0305c3420c None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-c4a95cb5-df6a-4eaf-982c-4b0305c3420c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:27:53.680558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:53.681678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:27:53.682953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:27:53.682953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:53.682953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:27:53.682953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 630/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:53.687459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57b1a4-ba7c-4825-b253-1fa1ad04573c None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:53.772858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:53.785230 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57b1a4-ba7c-4825-b253-1fa1ad04573c None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:53.817740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34cd65e-a778-4a79-ada1-0aba46c6ea37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:53.818357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cd65e-a778-4a79-ada1-0aba46c6ea37 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:53.818628 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cd65e-a778-4a79-ada1-0aba46c6ea37 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:53.818892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cd65e-a778-4a79-ada1-0aba46c6ea37 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:53.819517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34cd65e-a778-4a79-ada1-0aba46c6ea37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:53.819811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34cd65e-a778-4a79-ada1-0aba46c6ea37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:53.821154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34cd65e-a778-4a79-ada1-0aba46c6ea37 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:53.832189 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 627/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:53.837865 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a95cb5-df6a-4eaf-982c-4b0305c3420c None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:53.943121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a95cb5-df6a-4eaf-982c-4b0305c3420c None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:53.977424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833135f-49cc-451a-9afa-1c62685a9696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:53.978072 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833135f-49cc-451a-9afa-1c62685a9696 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:53.978344 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833135f-49cc-451a-9afa-1c62685a9696 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:53.978607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7833135f-49cc-451a-9afa-1c62685a9696 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:53.985672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833135f-49cc-451a-9afa-1c62685a9696 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:53.987162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7833135f-49cc-451a-9afa-1c62685a9696 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-7833135f-49cc-451a-9afa-1c62685a9696 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:27:53.998457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:54.000739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:54.001951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:27:54.001951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:27:54.001951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:27:54.001951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:27:54.001951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:27:54.001951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:27:54.001951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:27:54.001951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:27:54.001951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 631/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:27:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:54.005687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34cd65e-a778-4a79-ada1-0aba46c6ea37 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:54.103389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34cd65e-a778-4a79-ada1-0aba46c6ea37 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:54.135551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab29ec0-42b1-4656-aa42-1c71ce923550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:54.136134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab29ec0-42b1-4656-aa42-1c71ce923550 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:54.136406 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab29ec0-42b1-4656-aa42-1c71ce923550 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:54.136673 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab29ec0-42b1-4656-aa42-1c71ce923550 None admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:54.137349 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab29ec0-42b1-4656-aa42-1c71ce923550 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:54.137647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab29ec0-42b1-4656-aa42-1c71ce923550 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:54.139081 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab29ec0-42b1-4656-aa42-1c71ce923550 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:54.152292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 628/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:27:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:54.158859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833135f-49cc-451a-9afa-1c62685a9696 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:54.184118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:54.254824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7833135f-49cc-451a-9afa-1c62685a9696 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:54.289029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d67c56-1180-4fb2-aec1-6e0ed893787b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:54.289628 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d67c56-1180-4fb2-aec1-6e0ed893787b None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:54.289927 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d67c56-1180-4fb2-aec1-6e0ed893787b None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:54.290192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d67c56-1180-4fb2-aec1-6e0ed893787b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:54.290919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d67c56-1180-4fb2-aec1-6e0ed893787b None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=9975c0dab76f4ccbb4d2107e8e780ae5) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:54.291226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d67c56-1180-4fb2-aec1-6e0ed893787b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:54.292625 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d67c56-1180-4fb2-aec1-6e0ed893787b None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:54.298182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 632/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:27:54 2022] GET /identity/v3/role_assignments?user.id=9975c0dab76f4ccbb4d2107e8e780ae5&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:56.467421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc976ac0-1911-4794-905b-cf3c6f8a7918 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:56.467999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc976ac0-1911-4794-905b-cf3c6f8a7918 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:56.467999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc976ac0-1911-4794-905b-cf3c6f8a7918 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:56.469930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 629/1261] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:27:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:27:56.475247 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ea4b1-febc-4f96-8780-71c8d753fd38 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:56.475673 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ea4b1-febc-4f96-8780-71c8d753fd38 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:56.475781 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ea4b1-febc-4f96-8780-71c8d753fd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:56.494080 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-723ea4b1-febc-4f96-8780-71c8d753fd38 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:27:56.495272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-723ea4b1-febc-4f96-8780-71c8d753fd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:27:56.560904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 633/1262] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:27:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:27:56.841374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc976ac0-1911-4794-905b-cf3c6f8a7918 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:56.852875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:56.943706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc976ac0-1911-4794-905b-cf3c6f8a7918 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:56.977222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d284d17-3ac4-42ee-a172-addde3329aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:56.977848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d284d17-3ac4-42ee-a172-addde3329aec None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:56.978119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d284d17-3ac4-42ee-a172-addde3329aec None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:56.978379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d284d17-3ac4-42ee-a172-addde3329aec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:56.988743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d284d17-3ac4-42ee-a172-addde3329aec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-7d284d17-3ac4-42ee-a172-addde3329aec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:27:56.993648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:56.994789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:56.995883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:56.995883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:56.995883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:56.995883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:56.995883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:27:56.995883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:27:56.995883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:27:56.995883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:27:56.995883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:27:56.995883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:27:56.995883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:27:56.995883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:56.995883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 630/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:27:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:57.000991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ea4b1-febc-4f96-8780-71c8d753fd38 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:57.020312 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:57.101557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ea4b1-febc-4f96-8780-71c8d753fd38 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:57.135195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713c023-510a-420f-9290-a484a53559bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:57.135789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713c023-510a-420f-9290-a484a53559bb None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:57.136062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713c023-510a-420f-9290-a484a53559bb None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:57.136327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713c023-510a-420f-9290-a484a53559bb None admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:57.136977 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713c023-510a-420f-9290-a484a53559bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:57.137272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713c023-510a-420f-9290-a484a53559bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:57.138646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713c023-510a-420f-9290-a484a53559bb None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:57.147500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 634/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:27:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:57.153332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d284d17-3ac4-42ee-a172-addde3329aec None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:57.218445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:57.251314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d284d17-3ac4-42ee-a172-addde3329aec None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:57.285302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d92ad-bd58-4287-8be7-8f5ae761ba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:57.285940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d92ad-bd58-4287-8be7-8f5ae761ba01 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:57.286216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d92ad-bd58-4287-8be7-8f5ae761ba01 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:57.286478 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d92ad-bd58-4287-8be7-8f5ae761ba01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:57.296720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d92ad-bd58-4287-8be7-8f5ae761ba01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:57.298199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432d92ad-bd58-4287-8be7-8f5ae761ba01 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-432d92ad-bd58-4287-8be7-8f5ae761ba01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:27:57.311938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:57.313237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:27:57.314582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:27:57.314582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:57.314582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:57.317961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 631/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:57.318974 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713c023-510a-420f-9290-a484a53559bb None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:57.408434 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:57.427297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713c023-510a-420f-9290-a484a53559bb None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:57.459150 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500ba78-e57d-43ac-be46-b5a0ba9a41c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:57.459742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500ba78-e57d-43ac-be46-b5a0ba9a41c1 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:57.460016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500ba78-e57d-43ac-be46-b5a0ba9a41c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:57.460284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500ba78-e57d-43ac-be46-b5a0ba9a41c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:57.460930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500ba78-e57d-43ac-be46-b5a0ba9a41c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:57.461209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500ba78-e57d-43ac-be46-b5a0ba9a41c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:57.462588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500ba78-e57d-43ac-be46-b5a0ba9a41c1 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:57.476681 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 635/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:57.482529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d92ad-bd58-4287-8be7-8f5ae761ba01 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:57.581056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d92ad-bd58-4287-8be7-8f5ae761ba01 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:57.600551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:57.614281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c3ce1-4690-4cdb-93de-d9d20b3e166e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:57.619633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c3ce1-4690-4cdb-93de-d9d20b3e166e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:57.619633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c3ce1-4690-4cdb-93de-d9d20b3e166e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:57.619633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c3ce1-4690-4cdb-93de-d9d20b3e166e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:57.622432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c3ce1-4690-4cdb-93de-d9d20b3e166e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:57.623835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c3ce1-4690-4cdb-93de-d9d20b3e166e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-410c3ce1-4690-4cdb-93de-d9d20b3e166e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:27:57.635053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:57.636204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:57.637393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:57.637393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:57.637393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:27:57.637393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:27:57.637393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:27:57.637393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:27:57.637393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:27:57.637393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:27:57.637393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 05:27:57.637393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:57.637393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 632/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:27:57.642558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500ba78-e57d-43ac-be46-b5a0ba9a41c1 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:57.749379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500ba78-e57d-43ac-be46-b5a0ba9a41c1 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:57.783920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa8099-de83-40f4-87ae-e9006b3f8c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:57.784513 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa8099-de83-40f4-87ae-e9006b3f8c86 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:57.784784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa8099-de83-40f4-87ae-e9006b3f8c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:57.785046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa8099-de83-40f4-87ae-e9006b3f8c86 None admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:57.785751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa8099-de83-40f4-87ae-e9006b3f8c86 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:57.786073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa8099-de83-40f4-87ae-e9006b3f8c86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:57.787473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa8099-de83-40f4-87ae-e9006b3f8c86 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:57.795973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:57.804083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 636/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Sep 16 05:27:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:57.809881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c3ce1-4690-4cdb-93de-d9d20b3e166e None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:57.909787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c3ce1-4690-4cdb-93de-d9d20b3e166e None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:57.944088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90287d32-8240-4b28-affb-3cb1035606d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:57.944710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90287d32-8240-4b28-affb-3cb1035606d6 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:57.944982 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90287d32-8240-4b28-affb-3cb1035606d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:57.945248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90287d32-8240-4b28-affb-3cb1035606d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:57.956226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90287d32-8240-4b28-affb-3cb1035606d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:57.957684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90287d32-8240-4b28-affb-3cb1035606d6 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-90287d32-8240-4b28-affb-3cb1035606d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:27:57.972104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:57.973244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:27:57.974192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:27:57.974192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:27:57.974192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:57.974761 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 633/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:57.979284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa8099-de83-40f4-87ae-e9006b3f8c86 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:58.088057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa8099-de83-40f4-87ae-e9006b3f8c86 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:58.121939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827600d7-3d7b-4dbf-a2f1-ac03fd1d56d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:58.122536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827600d7-3d7b-4dbf-a2f1-ac03fd1d56d7 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:58.122807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827600d7-3d7b-4dbf-a2f1-ac03fd1d56d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:58.123071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827600d7-3d7b-4dbf-a2f1-ac03fd1d56d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:58.123699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827600d7-3d7b-4dbf-a2f1-ac03fd1d56d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:58.123992 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827600d7-3d7b-4dbf-a2f1-ac03fd1d56d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:58.126292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827600d7-3d7b-4dbf-a2f1-ac03fd1d56d7 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:58.138216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 637/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:58.145443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90287d32-8240-4b28-affb-3cb1035606d6 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:58.176937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:58.243500 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90287d32-8240-4b28-affb-3cb1035606d6 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:58.278869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5149733-56ec-40af-b98d-b519bc45ecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:58.279477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149733-56ec-40af-b98d-b519bc45ecb2 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:58.279748 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149733-56ec-40af-b98d-b519bc45ecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:58.280014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149733-56ec-40af-b98d-b519bc45ecb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:58.286573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5149733-56ec-40af-b98d-b519bc45ecb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:58.287999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5149733-56ec-40af-b98d-b519bc45ecb2 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-c5149733-56ec-40af-b98d-b519bc45ecb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:27:58.299093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:27:58.300153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:27:58.301272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:27:58.301272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:27:58.301272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:27:58.301272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:27:58.301272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:27:58.301272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:27:58.301272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:27:58.301272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:27:58.301272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 634/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:27:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:27:58.306305 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827600d7-3d7b-4dbf-a2f1-ac03fd1d56d7 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:58.392356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:58.432562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827600d7-3d7b-4dbf-a2f1-ac03fd1d56d7 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:58.465972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46262f-428e-4617-a324-2ec237012074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:58.466571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46262f-428e-4617-a324-2ec237012074 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:58.466848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46262f-428e-4617-a324-2ec237012074 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:58.467118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46262f-428e-4617-a324-2ec237012074 None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:58.467802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46262f-428e-4617-a324-2ec237012074 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:27:58.468129 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46262f-428e-4617-a324-2ec237012074 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:58.470202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46262f-428e-4617-a324-2ec237012074 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:58.481986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 638/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:27:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:27:58.489184 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5149733-56ec-40af-b98d-b519bc45ecb2 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:27:58.580238 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:27:58.588559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5149733-56ec-40af-b98d-b519bc45ecb2 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:27:58.620905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adb87b-02f2-4cb3-92d2-555dd9ce426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:27:58.621540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adb87b-02f2-4cb3-92d2-555dd9ce426c None admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:27:58.621811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adb87b-02f2-4cb3-92d2-555dd9ce426c None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:27:58.622097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adb87b-02f2-4cb3-92d2-555dd9ce426c None admin] PATH_INFO: `/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/9975c0dab76f4ccbb4d2107e8e780ae5/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:27:58.623928 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adb87b-02f2-4cb3-92d2-555dd9ce426c None admin] RBAC: Authorizing `identity:create_grant(project_id=29b2e27932984de79ee8dfbe95312ac2, user_id=9975c0dab76f4ccbb4d2107e8e780ae5, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:27:58.625455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85adb87b-02f2-4cb3-92d2-555dd9ce426c None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:27:58.674261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 635/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Fri Sep 16 05:27:58 2022] PUT /identity/v3/projects/29b2e27932984de79ee8dfbe95312ac2/users/9975c0dab76f4ccbb4d2107e8e780ae5/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:28:00.930458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29389c66-0668-4c25-90cc-f581381b1710 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:00.931022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29389c66-0668-4c25-90cc-f581381b1710 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:00.931022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29389c66-0668-4c25-90cc-f581381b1710 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:00.932326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 639/1274] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:28:00.938620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd756e-befe-4475-bbae-022d75d7a356 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:00.939069 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd756e-befe-4475-bbae-022d75d7a356 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:00.939174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd756e-befe-4475-bbae-022d75d7a356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:00.961283 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-1fdd756e-befe-4475-bbae-022d75d7a356 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:28:00.962829 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1fdd756e-befe-4475-bbae-022d75d7a356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:28:01.037693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 636/1275] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:28:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:28:01.334016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29389c66-0668-4c25-90cc-f581381b1710 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:01.434067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29389c66-0668-4c25-90cc-f581381b1710 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:01.451110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:01.468141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9aa5e-8ca4-4b03-a5c8-400fc48ce2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:01.468757 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9aa5e-8ca4-4b03-a5c8-400fc48ce2ad None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:01.469278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9aa5e-8ca4-4b03-a5c8-400fc48ce2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:01.469419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9aa5e-8ca4-4b03-a5c8-400fc48ce2ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:01.485143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9aa5e-8ca4-4b03-a5c8-400fc48ce2ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-09c9aa5e-8ca4-4b03-a5c8-400fc48ce2ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 16 05:28:01.490076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:28:01.491313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:28:01.492169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:28:01.492169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:28:01.492169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:28:01.492169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:28:01.492169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 16 05:28:01.492169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 16 05:28:01.492169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 16 05:28:01.492169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 16 05:28:01.492169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 16 05:28:01.492169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 16 05:28:01.492169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 16 05:28:01.492169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:28:01.492169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 640/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Fri Sep 16 05:28:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:28:01.497189 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd756e-befe-4475-bbae-022d75d7a356 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:01.577741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:01.595826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd756e-befe-4475-bbae-022d75d7a356 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:01.631671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81820f88-d7fe-4d99-b039-22f7df9583e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:01.632325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81820f88-d7fe-4d99-b039-22f7df9583e3 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:01.632597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81820f88-d7fe-4d99-b039-22f7df9583e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:01.632855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81820f88-d7fe-4d99-b039-22f7df9583e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:01.633503 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81820f88-d7fe-4d99-b039-22f7df9583e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:28:01.633795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81820f88-d7fe-4d99-b039-22f7df9583e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:01.635164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81820f88-d7fe-4d99-b039-22f7df9583e3 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:01.643343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 637/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Fri Sep 16 05:28:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:28:01.649376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9aa5e-8ca4-4b03-a5c8-400fc48ce2ad None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:01.748858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9aa5e-8ca4-4b03-a5c8-400fc48ce2ad None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:01.782268 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fae7f7-160b-46b3-a905-6eef46bc69c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:01.782878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fae7f7-160b-46b3-a905-6eef46bc69c7 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:01.783148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fae7f7-160b-46b3-a905-6eef46bc69c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:01.783412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fae7f7-160b-46b3-a905-6eef46bc69c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:01.793197 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fae7f7-160b-46b3-a905-6eef46bc69c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:01.794640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fae7f7-160b-46b3-a905-6eef46bc69c7 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-c9fae7f7-160b-46b3-a905-6eef46bc69c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:28:01.810663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:28:01.811798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:28:01.813533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:28:01.813533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:28:01.813533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:28:01.813533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 641/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:28:01.818277 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81820f88-d7fe-4d99-b039-22f7df9583e3 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:01.918522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81820f88-d7fe-4d99-b039-22f7df9583e3 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:01.952907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb068f6-0629-4f4a-9f7e-0099a93cdd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:01.953629 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb068f6-0629-4f4a-9f7e-0099a93cdd68 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:01.953902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb068f6-0629-4f4a-9f7e-0099a93cdd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:01.954163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb068f6-0629-4f4a-9f7e-0099a93cdd68 None admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:01.954793 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb068f6-0629-4f4a-9f7e-0099a93cdd68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:28:01.959539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb068f6-0629-4f4a-9f7e-0099a93cdd68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:01.959539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb068f6-0629-4f4a-9f7e-0099a93cdd68 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:01.967991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 638/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:28:01.974500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fae7f7-160b-46b3-a905-6eef46bc69c7 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:02.014259 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:02.085280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fae7f7-160b-46b3-a905-6eef46bc69c7 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:02.118927 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d57f6b-6c68-4dce-be64-4dd811e6fdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:02.119542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d57f6b-6c68-4dce-be64-4dd811e6fdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:02.119813 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d57f6b-6c68-4dce-be64-4dd811e6fdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:02.120117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d57f6b-6c68-4dce-be64-4dd811e6fdf1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:02.128032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d57f6b-6c68-4dce-be64-4dd811e6fdf1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:02.129440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d57f6b-6c68-4dce-be64-4dd811e6fdf1 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-63d57f6b-6c68-4dce-be64-4dd811e6fdf1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 16 05:28:02.140368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:28:02.142334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:28:02.143674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:28:02.143674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:28:02.143674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 16 05:28:02.143674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 16 05:28:02.143674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 16 05:28:02.143674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 16 05:28:02.143674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 16 05:28:02.143674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 16 05:28:02.143674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 16 05:28:02.143674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:28:02.143674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 642/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:28:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 16 05:28:02.147564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb068f6-0629-4f4a-9f7e-0099a93cdd68 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:02.151314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:02.243672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb068f6-0629-4f4a-9f7e-0099a93cdd68 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:02.279361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00ebee-8047-4a2c-b8c0-dca5171bfa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:02.279956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00ebee-8047-4a2c-b8c0-dca5171bfa9f None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:02.280230 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00ebee-8047-4a2c-b8c0-dca5171bfa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:02.280497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00ebee-8047-4a2c-b8c0-dca5171bfa9f None admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:02.281262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00ebee-8047-4a2c-b8c0-dca5171bfa9f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:28:02.281620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00ebee-8047-4a2c-b8c0-dca5171bfa9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:02.283066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00ebee-8047-4a2c-b8c0-dca5171bfa9f None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:02.301114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 639/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Fri Sep 16 05:28:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:28:02.307331 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d57f6b-6c68-4dce-be64-4dd811e6fdf1 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:02.398511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:02.408016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d57f6b-6c68-4dce-be64-4dd811e6fdf1 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:02.441756 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82253fcb-495d-45b1-8afe-f2f2d8b898ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:02.442383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82253fcb-495d-45b1-8afe-f2f2d8b898ec None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:02.442661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82253fcb-495d-45b1-8afe-f2f2d8b898ec None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:02.442931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82253fcb-495d-45b1-8afe-f2f2d8b898ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:02.509166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82253fcb-495d-45b1-8afe-f2f2d8b898ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:02.510565 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82253fcb-495d-45b1-8afe-f2f2d8b898ec None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:02.596296 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:02.606160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 643/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Fri Sep 16 05:28:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 16 05:28:02.612671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00ebee-8047-4a2c-b8c0-dca5171bfa9f None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:02.674420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:02.711971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00ebee-8047-4a2c-b8c0-dca5171bfa9f None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:02.747311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63836446-bf3d-4da9-8fb7-7d3235588a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:02.747924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63836446-bf3d-4da9-8fb7-7d3235588a73 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:02.748195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63836446-bf3d-4da9-8fb7-7d3235588a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:02.748460 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63836446-bf3d-4da9-8fb7-7d3235588a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:02.758979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63836446-bf3d-4da9-8fb7-7d3235588a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:02.760505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63836446-bf3d-4da9-8fb7-7d3235588a73 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-63836446-bf3d-4da9-8fb7-7d3235588a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 16 05:28:02.775836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:28:02.777047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 16 05:28:02.778150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 16 05:28:02.778150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 16 05:28:02.778150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:28:02.778150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 640/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:28:02.783506 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82253fcb-495d-45b1-8afe-f2f2d8b898ec None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:02.880933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82253fcb-495d-45b1-8afe-f2f2d8b898ec None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:02.916230 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06e1d8-f4e5-4f77-9e4d-4f6687efb978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:02.916836 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06e1d8-f4e5-4f77-9e4d-4f6687efb978 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:02.917108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06e1d8-f4e5-4f77-9e4d-4f6687efb978 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:02.917375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06e1d8-f4e5-4f77-9e4d-4f6687efb978 None admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:02.918033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06e1d8-f4e5-4f77-9e4d-4f6687efb978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:28:02.918327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06e1d8-f4e5-4f77-9e4d-4f6687efb978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:02.919713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06e1d8-f4e5-4f77-9e4d-4f6687efb978 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:02.931154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 644/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Fri Sep 16 05:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:28:02.936851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63836446-bf3d-4da9-8fb7-7d3235588a73 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:03.034314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63836446-bf3d-4da9-8fb7-7d3235588a73 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:03.048115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:03.067963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd1d66-5685-45c5-9394-299215f47b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:03.068651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd1d66-5685-45c5-9394-299215f47b7b None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:03.068927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd1d66-5685-45c5-9394-299215f47b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:03.069193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd1d66-5685-45c5-9394-299215f47b7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:03.075642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd1d66-5685-45c5-9394-299215f47b7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:03.077057 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dd1d66-5685-45c5-9394-299215f47b7b None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application [None req-c4dd1d66-5685-45c5-9394-299215f47b7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 16 05:28:03.088656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application with Lock( Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:28:03.096417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:28:03.097530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 16 05:28:03.097530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 16 05:28:03.097530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 16 05:28:03.097530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 16 05:28:03.097530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 16 05:28:03.097530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 16 05:28:03.097530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 16 05:28:03.097530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: ERROR keystone.server.flask.application Sep 16 05:28:03.097530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 641/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Fri Sep 16 05:28:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:28:03.098209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06e1d8-f4e5-4f77-9e4d-4f6687efb978 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:03.146884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:03.195029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06e1d8-f4e5-4f77-9e4d-4f6687efb978 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:03.226668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5616a58-21f3-4812-ad5b-8aab37adeed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:03.227264 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5616a58-21f3-4812-ad5b-8aab37adeed5 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:03.227534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5616a58-21f3-4812-ad5b-8aab37adeed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:03.227794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5616a58-21f3-4812-ad5b-8aab37adeed5 None admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:03.228461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5616a58-21f3-4812-ad5b-8aab37adeed5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:28:03.228753 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5616a58-21f3-4812-ad5b-8aab37adeed5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:03.230216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5616a58-21f3-4812-ad5b-8aab37adeed5 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:03.242337 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 645/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Fri Sep 16 05:28:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:28:03.248437 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd1d66-5685-45c5-9394-299215f47b7b None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:03.345910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd1d66-5685-45c5-9394-299215f47b7b None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:03.378955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ddf3d-a6e6-4cad-b049-f4c4527c9108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:03.379540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ddf3d-a6e6-4cad-b049-f4c4527c9108 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:03.379812 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ddf3d-a6e6-4cad-b049-f4c4527c9108 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:03.380070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ddf3d-a6e6-4cad-b049-f4c4527c9108 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:03.380765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ddf3d-a6e6-4cad-b049-f4c4527c9108 None admin] RBAC: Adding query filter params (role.id=9c3e3861a9ce4828829f793b31a275f0, scope.project.id=29b2e27932984de79ee8dfbe95312ac2, user.id=9975c0dab76f4ccbb4d2107e8e780ae5) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:28:03.381068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ddf3d-a6e6-4cad-b049-f4c4527c9108 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:03.382488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ddf3d-a6e6-4cad-b049-f4c4527c9108 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:03.388038 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 642/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Fri Sep 16 05:28:03 2022] GET /identity/v3/role_assignments?user.id=9975c0dab76f4ccbb4d2107e8e780ae5&scope.project.id=29b2e27932984de79ee8dfbe95312ac2&role.id=9c3e3861a9ce4828829f793b31a275f0 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:28:05.632398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef7f9a-7135-4be0-b015-6fc9212df170 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:05.632952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef7f9a-7135-4be0-b015-6fc9212df170 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:05.633054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef7f9a-7135-4be0-b015-6fc9212df170 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:05.634340 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 646/1288] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:28:05.639364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bd6a9-1e7f-4a37-9360-f5b1c7a510ff None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:05.639756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bd6a9-1e7f-4a37-9360-f5b1c7a510ff None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:05.639911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bd6a9-1e7f-4a37-9360-f5b1c7a510ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:05.659310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-d01bd6a9-1e7f-4a37-9360-f5b1c7a510ff None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:28:05.660488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d01bd6a9-1e7f-4a37-9360-f5b1c7a510ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:28:05.690255 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:05.726567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 643/1289] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:28:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:28:06.003163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef7f9a-7135-4be0-b015-6fc9212df170 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:06.111665 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef7f9a-7135-4be0-b015-6fc9212df170 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:06.122025 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:06.144528 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764603f0-60fc-4777-bfa5-63ef9338db7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:06.145139 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764603f0-60fc-4777-bfa5-63ef9338db7a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:06.145407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764603f0-60fc-4777-bfa5-63ef9338db7a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:06.145668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764603f0-60fc-4777-bfa5-63ef9338db7a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:06.146260 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764603f0-60fc-4777-bfa5-63ef9338db7a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:06.147595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764603f0-60fc-4777-bfa5-63ef9338db7a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-764603f0-60fc-4777-bfa5-63ef9338db7a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:28:06.156729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:28:06.162229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:28:06.163313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 647/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 16 05:28:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:28:06.164368 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bd6a9-1e7f-4a37-9360-f5b1c7a510ff None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:06.260852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bd6a9-1e7f-4a37-9360-f5b1c7a510ff None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:06.294609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cd953-77c6-4430-bb2d-eabd5251486e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:06.295202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cd953-77c6-4430-bb2d-eabd5251486e None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:06.295498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cd953-77c6-4430-bb2d-eabd5251486e None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:06.295763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cd953-77c6-4430-bb2d-eabd5251486e None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:06.296375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cd953-77c6-4430-bb2d-eabd5251486e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:28:06.296668 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cd953-77c6-4430-bb2d-eabd5251486e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:06.299000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cd953-77c6-4430-bb2d-eabd5251486e None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:06.303616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 644/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Sep 16 05:28:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:28:06.309531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764603f0-60fc-4777-bfa5-63ef9338db7a None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:06.407786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764603f0-60fc-4777-bfa5-63ef9338db7a None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:06.441054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b835e-b968-4e89-a6ef-b0a6b079be5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:06.441649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b835e-b968-4e89-a6ef-b0a6b079be5a None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:06.441952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b835e-b968-4e89-a6ef-b0a6b079be5a None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:06.442224 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b835e-b968-4e89-a6ef-b0a6b079be5a None admin] PATH_INFO: `/v3/services` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:06.442844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b835e-b968-4e89-a6ef-b0a6b079be5a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:28:06.443140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b835e-b968-4e89-a6ef-b0a6b079be5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:06.444493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b835e-b968-4e89-a6ef-b0a6b079be5a None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:06.448798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 648/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Sep 16 05:28:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:28:08.591469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4996f-aff8-4d55-8b23-c21b23bfd0a1 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:08.592019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4996f-aff8-4d55-8b23-c21b23bfd0a1 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:08.592019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4996f-aff8-4d55-8b23-c21b23bfd0a1 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:08.593310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 645/1293] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:28:08.598641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78fd87-2385-4697-aed5-88407a4020bf None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:08.599048 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78fd87-2385-4697-aed5-88407a4020bf None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:08.599179 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78fd87-2385-4697-aed5-88407a4020bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:08.617384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-ad78fd87-2385-4697-aed5-88407a4020bf None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:28:08.618583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ad78fd87-2385-4697-aed5-88407a4020bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:28:08.647478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:08.682823 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 649/1294] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:28:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:28:08.949803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4996f-aff8-4d55-8b23-c21b23bfd0a1 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:08.980742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:09.048856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4996f-aff8-4d55-8b23-c21b23bfd0a1 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:09.083323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8988e50-3dea-4875-a321-a23b183699a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:09.083915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8988e50-3dea-4875-a321-a23b183699a7 None admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:09.084188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8988e50-3dea-4875-a321-a23b183699a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:09.084455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8988e50-3dea-4875-a321-a23b183699a7 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:09.085104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8988e50-3dea-4875-a321-a23b183699a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:09.086512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8988e50-3dea-4875-a321-a23b183699a7 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:09.165432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 646/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Fri Sep 16 05:28:08 2022] POST /identity/v3/services => generated 246 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:28:11.325869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4b563-021a-4a68-8cce-78a25a1731f1 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:11.326396 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4b563-021a-4a68-8cce-78a25a1731f1 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:11.326396 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4b563-021a-4a68-8cce-78a25a1731f1 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:11.327652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 650/1296] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:28:11.333243 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675e09d-55c3-45c3-be9c-b72d7a005ca8 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:11.333768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675e09d-55c3-45c3-be9c-b72d7a005ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:11.333768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675e09d-55c3-45c3-be9c-b72d7a005ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:11.352480 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-5675e09d-55c3-45c3-be9c-b72d7a005ca8 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:28:11.353665 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5675e09d-55c3-45c3-be9c-b72d7a005ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:28:11.425067 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 647/1297] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:28:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 16 05:28:11.707727 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4b563-021a-4a68-8cce-78a25a1731f1 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:11.807751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4b563-021a-4a68-8cce-78a25a1731f1 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:11.841707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527b098-d9c2-4814-9fa0-3e7a0615d275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:11.842359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527b098-d9c2-4814-9fa0-3e7a0615d275 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:11.842634 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527b098-d9c2-4814-9fa0-3e7a0615d275 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:11.842900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2527b098-d9c2-4814-9fa0-3e7a0615d275 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:11.843502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527b098-d9c2-4814-9fa0-3e7a0615d275 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:11.844948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2527b098-d9c2-4814-9fa0-3e7a0615d275 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-2527b098-d9c2-4814-9fa0-3e7a0615d275 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:28:11.853985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:28:11.858385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:28:11.859368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 651/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 16 05:28:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:28:11.861459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675e09d-55c3-45c3-be9c-b72d7a005ca8 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:11.879749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:11.958942 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675e09d-55c3-45c3-be9c-b72d7a005ca8 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:11.991213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99b362-95f5-4ff5-9985-240e05f03e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:11.991806 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99b362-95f5-4ff5-9985-240e05f03e15 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:11.992099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99b362-95f5-4ff5-9985-240e05f03e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:11.992381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99b362-95f5-4ff5-9985-240e05f03e15 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:11.993003 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99b362-95f5-4ff5-9985-240e05f03e15 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:28:11.993497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99b362-95f5-4ff5-9985-240e05f03e15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:11.995781 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99b362-95f5-4ff5-9985-240e05f03e15 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:12.001406 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 648/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Sep 16 05:28:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:28:12.007402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527b098-d9c2-4814-9fa0-3e7a0615d275 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:12.038569 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:12.106413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2527b098-d9c2-4814-9fa0-3e7a0615d275 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:12.139948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9431f4-7b98-40ae-b39e-40dca06218db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:12.140576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9431f4-7b98-40ae-b39e-40dca06218db None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:12.141034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9431f4-7b98-40ae-b39e-40dca06218db None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:12.141271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9431f4-7b98-40ae-b39e-40dca06218db None admin] PATH_INFO: `/v3/endpoints` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:12.142070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9431f4-7b98-40ae-b39e-40dca06218db None admin] RBAC: Adding query filter params (interface=public, service_id=2bda40b1e465448b80551a6515f5cdbe, region_id=RegionOne) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:28:12.142375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9431f4-7b98-40ae-b39e-40dca06218db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:12.143729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9431f4-7b98-40ae-b39e-40dca06218db None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:12.148610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 652/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Fri Sep 16 05:28:12 2022] GET /identity/v3/endpoints?service_id=2bda40b1e465448b80551a6515f5cdbe&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:28:12.154405 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99b362-95f5-4ff5-9985-240e05f03e15 None admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:12.234652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:12.252474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99b362-95f5-4ff5-9985-240e05f03e15 None admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:12.293897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804c07c-10d3-47f1-b7c3-e7d8baa7af96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:12.294453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804c07c-10d3-47f1-b7c3-e7d8baa7af96 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:12.294724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804c07c-10d3-47f1-b7c3-e7d8baa7af96 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:12.294985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804c07c-10d3-47f1-b7c3-e7d8baa7af96 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:12.295578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804c07c-10d3-47f1-b7c3-e7d8baa7af96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:12.299201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804c07c-10d3-47f1-b7c3-e7d8baa7af96 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:12.304732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 649/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Fri Sep 16 05:28:12 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:28:14.491656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d85cfb-1c5c-442c-85e2-827e746d44b2 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:14.492194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d85cfb-1c5c-442c-85e2-827e746d44b2 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:14.492194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d85cfb-1c5c-442c-85e2-827e746d44b2 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:14.493518 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 653/1302] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:28:14.498792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d91cf8-23f9-4ab9-9703-02b0da314e25 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:14.499197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d91cf8-23f9-4ab9-9703-02b0da314e25 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:14.501931 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d91cf8-23f9-4ab9-9703-02b0da314e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:14.518195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-04d91cf8-23f9-4ab9-9703-02b0da314e25 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:28:14.519368 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-04d91cf8-23f9-4ab9-9703-02b0da314e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:28:14.584020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 650/1303] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:28:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 16 05:28:14.857459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d85cfb-1c5c-442c-85e2-827e746d44b2 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:14.958209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:14.964741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d85cfb-1c5c-442c-85e2-827e746d44b2 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:14.999116 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4ff1a-6f63-4977-8875-3c2d12421307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:14.999797 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4ff1a-6f63-4977-8875-3c2d12421307 None admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:15.000297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4ff1a-6f63-4977-8875-3c2d12421307 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:15.000568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af4ff1a-6f63-4977-8875-3c2d12421307 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:15.001140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4ff1a-6f63-4977-8875-3c2d12421307 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:15.002512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af4ff1a-6f63-4977-8875-3c2d12421307 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application [None req-7af4ff1a-6f63-4977-8875-3c2d12421307 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 16 05:28:15.011081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 16 05:28:15.014302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.get_or_create( Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application with Lock( Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self._enter() Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application return self.creator() Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application created_value = creator( Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: ERROR keystone.server.flask.application Sep 16 05:28:15.016030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 654/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Fri Sep 16 05:28:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 16 05:28:15.019260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d91cf8-23f9-4ab9-9703-02b0da314e25 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:15.094212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:15.116137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d91cf8-23f9-4ab9-9703-02b0da314e25 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:15.150605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb94ad-7bed-4e41-a6d4-2b90078d46d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:15.151202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb94ad-7bed-4e41-a6d4-2b90078d46d2 None admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:15.151472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb94ad-7bed-4e41-a6d4-2b90078d46d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:15.151735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb94ad-7bed-4e41-a6d4-2b90078d46d2 None admin] PATH_INFO: `/v3/services` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:15.152413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb94ad-7bed-4e41-a6d4-2b90078d46d2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:28:15.152794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb94ad-7bed-4e41-a6d4-2b90078d46d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:15.154171 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb94ad-7bed-4e41-a6d4-2b90078d46d2 None admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:15.159780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 651/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Fri Sep 16 05:28:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:28:15.165604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4ff1a-6f63-4977-8875-3c2d12421307 None admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:28:15.278548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af4ff1a-6f63-4977-8875-3c2d12421307 None admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:28:15.311418 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9197ea59-f28f-4317-bd6a-d84b42f98fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:28:15.312009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197ea59-f28f-4317-bd6a-d84b42f98fb9 None admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:28:15.313124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197ea59-f28f-4317-bd6a-d84b42f98fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:28:15.313124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9197ea59-f28f-4317-bd6a-d84b42f98fb9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:28:15.313300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197ea59-f28f-4317-bd6a-d84b42f98fb9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:28:15.314654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9197ea59-f28f-4317-bd6a-d84b42f98fb9 None admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:28:15.399761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 655/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:28:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:28:25.156148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=30 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:28:25.395664 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:05.529095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af4770-9786-469f-a63e-284ce53772e2 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:05.529575 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af4770-9786-469f-a63e-284ce53772e2 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:05.529669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af4770-9786-469f-a63e-284ce53772e2 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:05.535129 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 652/1307] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:34:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:05.536571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f1397-1e82-4f19-824e-cd72b190cb60 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:05.537133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f1397-1e82-4f19-824e-cd72b190cb60 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:05.537133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f1397-1e82-4f19-824e-cd72b190cb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:05.683619 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-0e9f1397-1e82-4f19-824e-cd72b190cb60 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:05.684855 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0e9f1397-1e82-4f19-824e-cd72b190cb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:05.901847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 656/1308] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:34:06.204823 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0b48f-ba3b-49d6-9077-4313bbd4b3b7 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:06.205508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0b48f-ba3b-49d6-9077-4313bbd4b3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:06.206037 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0b48f-ba3b-49d6-9077-4313bbd4b3b7 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:06.207636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 653/1309] 10.222.0.49 () {54 vars in 965 bytes} [Fri Sep 16 05:34:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:06.215279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7ecbf-38f9-4655-9b0f-ad60e2cab9b0 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:06.215952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7ecbf-38f9-4655-9b0f-ad60e2cab9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:06.216437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7ecbf-38f9-4655-9b0f-ad60e2cab9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:06.310278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-52c7ecbf-38f9-4655-9b0f-ad60e2cab9b0 None None] MFA Rules not processed for user `3e923be9aeed43728690d24d85dbe062`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:06.311490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-52c7ecbf-38f9-4655-9b0f-ad60e2cab9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:06.334036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:06.400018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 657/1310] 10.222.0.49 () {58 vars in 1062 bytes} [Fri Sep 16 05:34:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:34:06.409696 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0b48f-ba3b-49d6-9077-4313bbd4b3b7 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:34:06.411472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f3a0b48f-ba3b-49d6-9077-4313bbd4b3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:06.545907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0b48f-ba3b-49d6-9077-4313bbd4b3b7 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:34:06.633431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989d2ec-4d18-4ea8-9fcf-a12e86fe9d2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:34:06.634072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989d2ec-4d18-4ea8-9fcf-a12e86fe9d2c service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:06.634350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989d2ec-4d18-4ea8-9fcf-a12e86fe9d2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:06.634621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989d2ec-4d18-4ea8-9fcf-a12e86fe9d2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:06.636805 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a989d2ec-4d18-4ea8-9fcf-a12e86fe9d2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:06.706454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:06.773640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989d2ec-4d18-4ea8-9fcf-a12e86fe9d2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:34:06.775803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a989d2ec-4d18-4ea8-9fcf-a12e86fe9d2c service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:34:06.801217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 654/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:34:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:34:09.355419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b6e2c-788d-4852-b503-f546ef93c726 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:09.355926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b6e2c-788d-4852-b503-f546ef93c726 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:09.355926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b6e2c-788d-4852-b503-f546ef93c726 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:09.357225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 658/1312] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:34:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:09.362582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212213eb-92b7-4f50-9131-f8a6db1125b2 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:09.362995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212213eb-92b7-4f50-9131-f8a6db1125b2 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:09.363116 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212213eb-92b7-4f50-9131-f8a6db1125b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:09.444333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-212213eb-92b7-4f50-9131-f8a6db1125b2 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:09.445524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-212213eb-92b7-4f50-9131-f8a6db1125b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:09.540442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 655/1313] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:34:09.825323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b6e2c-788d-4852-b503-f546ef93c726 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:34:09.826878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-925b6e2c-788d-4852-b503-f546ef93c726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:09.929950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b6e2c-788d-4852-b503-f546ef93c726 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:34:10.001208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:10.003503 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f9fd0-0747-419a-b533-63b100d5231a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:34:10.004105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f9fd0-0747-419a-b533-63b100d5231a service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:10.004385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f9fd0-0747-419a-b533-63b100d5231a service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:10.004649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f9fd0-0747-419a-b533-63b100d5231a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:10.006259 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-542f9fd0-0747-419a-b533-63b100d5231a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:10.125678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f9fd0-0747-419a-b533-63b100d5231a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:34:10.127135 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f9fd0-0747-419a-b533-63b100d5231a service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:34:10.145389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 659/1314] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:34:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:34:12.835174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aeebc8-8cb4-43f6-aa23-89a630d07845 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:12.836483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aeebc8-8cb4-43f6-aa23-89a630d07845 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:12.836483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aeebc8-8cb4-43f6-aa23-89a630d07845 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:12.837085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 656/1315] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:12.842377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686724f-62f0-4d88-b575-63cf5dd520da None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:12.842815 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686724f-62f0-4d88-b575-63cf5dd520da None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:12.842912 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7686724f-62f0-4d88-b575-63cf5dd520da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:12.909129 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-7686724f-62f0-4d88-b575-63cf5dd520da None None] MFA Rules not processed for user `fd426537f3dc4593941437dc64e7c961`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:12.910353 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7686724f-62f0-4d88-b575-63cf5dd520da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:12.961577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:13.006932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 660/1316] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:34:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:34:13.305234 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aeebc8-8cb4-43f6-aa23-89a630d07845 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:34:13.307303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-32aeebc8-8cb4-43f6-aa23-89a630d07845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:13.316994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:13.415119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aeebc8-8cb4-43f6-aa23-89a630d07845 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:34:13.487973 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667cfe40-8de4-4c8f-998f-a7d8cbf9388f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:34:13.488566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cfe40-8de4-4c8f-998f-a7d8cbf9388f service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:13.488841 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cfe40-8de4-4c8f-998f-a7d8cbf9388f service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:13.489111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cfe40-8de4-4c8f-998f-a7d8cbf9388f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:13.490723 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-667cfe40-8de4-4c8f-998f-a7d8cbf9388f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:13.552683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:13.598091 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667cfe40-8de4-4c8f-998f-a7d8cbf9388f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:34:13.599495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667cfe40-8de4-4c8f-998f-a7d8cbf9388f service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:34:13.617905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 657/1317] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:34:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:34:16.519572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b38dff-c5b7-4009-8391-604f6867738d None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:16.520137 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b38dff-c5b7-4009-8391-604f6867738d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:16.520137 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b38dff-c5b7-4009-8391-604f6867738d None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:16.521464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 661/1318] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:16.527642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bc8d9-2849-4d9d-8233-227470fd02ce None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:16.528089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bc8d9-2849-4d9d-8233-227470fd02ce None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:16.528185 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bc8d9-2849-4d9d-8233-227470fd02ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:16.594443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-be4bc8d9-2849-4d9d-8233-227470fd02ce None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:16.595660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-be4bc8d9-2849-4d9d-8233-227470fd02ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:16.690822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 658/1319] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:34:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:34:16.984827 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b38dff-c5b7-4009-8391-604f6867738d None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:34:16.986386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-35b38dff-c5b7-4009-8391-604f6867738d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:17.090457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b38dff-c5b7-4009-8391-604f6867738d None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:34:17.161671 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ad070-19a9-4f1d-847a-3a2d99430cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:34:17.162306 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ad070-19a9-4f1d-847a-3a2d99430cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:17.162574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ad070-19a9-4f1d-847a-3a2d99430cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:17.162841 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ad070-19a9-4f1d-847a-3a2d99430cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:17.164392 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4e8ad070-19a9-4f1d-847a-3a2d99430cc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:17.171916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:17.269097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ad070-19a9-4f1d-847a-3a2d99430cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:34:17.270564 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ad070-19a9-4f1d-847a-3a2d99430cc9 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:34:17.287901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 662/1320] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:34:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:34:20.421078 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6310a-6585-473c-a5ac-96705ccdc32f None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:20.421642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6310a-6585-473c-a5ac-96705ccdc32f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:20.421642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6310a-6585-473c-a5ac-96705ccdc32f None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:20.423092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 659/1321] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:20.428588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45ede4-0656-4ee1-92b9-d0165d43fa87 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:20.429034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45ede4-0656-4ee1-92b9-d0165d43fa87 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:20.429136 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df45ede4-0656-4ee1-92b9-d0165d43fa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:20.494725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-df45ede4-0656-4ee1-92b9-d0165d43fa87 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:20.495781 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-df45ede4-0656-4ee1-92b9-d0165d43fa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:20.559591 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:20.603246 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 663/1322] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:34:20.886758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6310a-6585-473c-a5ac-96705ccdc32f None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:34:20.888379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-93e6310a-6585-473c-a5ac-96705ccdc32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:20.909048 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:20.993517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6310a-6585-473c-a5ac-96705ccdc32f None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:34:21.067188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed5ce8-fb1b-49fc-aaad-181779e3a8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:34:21.067791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed5ce8-fb1b-49fc-aaad-181779e3a8dc service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:21.068062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed5ce8-fb1b-49fc-aaad-181779e3a8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:21.068326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed5ce8-fb1b-49fc-aaad-181779e3a8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:21.069926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a3ed5ce8-fb1b-49fc-aaad-181779e3a8dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:21.152814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:21.195323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed5ce8-fb1b-49fc-aaad-181779e3a8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:34:21.196773 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed5ce8-fb1b-49fc-aaad-181779e3a8dc service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:34:21.214785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 660/1323] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:34:23.427075 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62138e46-90be-4edf-9f9e-90c9a6f588a7 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:23.427623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62138e46-90be-4edf-9f9e-90c9a6f588a7 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:23.427623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62138e46-90be-4edf-9f9e-90c9a6f588a7 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:23.428933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 664/1324] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:23.434839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e2f4b-d291-462c-9de9-675ee7619613 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:23.435235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e2f4b-d291-462c-9de9-675ee7619613 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:23.435370 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e2f4b-d291-462c-9de9-675ee7619613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:23.510270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-b44e2f4b-d291-462c-9de9-675ee7619613 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:23.511570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b44e2f4b-d291-462c-9de9-675ee7619613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:23.608113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 661/1325] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:34:23.893567 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62138e46-90be-4edf-9f9e-90c9a6f588a7 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:34:23.895124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-62138e46-90be-4edf-9f9e-90c9a6f588a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:24.000162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62138e46-90be-4edf-9f9e-90c9a6f588a7 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:34:24.074184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb64cc3-b257-42c6-bc07-57d64c0640ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:34:24.074791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb64cc3-b257-42c6-bc07-57d64c0640ad service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:24.075061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb64cc3-b257-42c6-bc07-57d64c0640ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:24.075320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb64cc3-b257-42c6-bc07-57d64c0640ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:24.077083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5fb64cc3-b257-42c6-bc07-57d64c0640ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:24.095180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:24.192163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb64cc3-b257-42c6-bc07-57d64c0640ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:34:24.193593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb64cc3-b257-42c6-bc07-57d64c0640ad service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:34:24.211892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 665/1326] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:34:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:34:26.470648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987d096-b578-4cfd-957a-f85c91dcd8b3 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:26.471164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987d096-b578-4cfd-957a-f85c91dcd8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:26.471164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987d096-b578-4cfd-957a-f85c91dcd8b3 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:26.472518 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 662/1327] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:26.477786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99d474-057d-4af6-8fc0-2786eed96ff9 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:26.478251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99d474-057d-4af6-8fc0-2786eed96ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:26.478925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99d474-057d-4af6-8fc0-2786eed96ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:26.551047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-ca99d474-057d-4af6-8fc0-2786eed96ff9 None None] MFA Rules not processed for user `fd426537f3dc4593941437dc64e7c961`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:26.552156 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ca99d474-057d-4af6-8fc0-2786eed96ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:26.617158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:26.647676 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 666/1328] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:34:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:34:26.951688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152efc73-264d-493c-982a-ae7c4e72b96b None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:26.951962 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152efc73-264d-493c-982a-ae7c4e72b96b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:26.953163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152efc73-264d-493c-982a-ae7c4e72b96b None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:26.954568 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 663/1329] 10.222.0.49 () {54 vars in 965 bytes} [Fri Sep 16 05:34:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:26.961067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76fc51-026e-47f0-90b9-e3773f276441 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:26.961365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76fc51-026e-47f0-90b9-e3773f276441 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:26.961602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76fc51-026e-47f0-90b9-e3773f276441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:27.044644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-aa76fc51-026e-47f0-90b9-e3773f276441 None None] MFA Rules not processed for user `3e923be9aeed43728690d24d85dbe062`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:27.045944 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-aa76fc51-026e-47f0-90b9-e3773f276441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:27.135786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 667/1330] 10.222.0.49 () {58 vars in 1062 bytes} [Fri Sep 16 05:34:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:34:27.144655 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152efc73-264d-493c-982a-ae7c4e72b96b None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:34:27.146454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-152efc73-264d-493c-982a-ae7c4e72b96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:27.181755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:27.259199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152efc73-264d-493c-982a-ae7c4e72b96b None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:34:27.339354 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1da09-cef6-43c7-925b-edd8f224ce3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:34:27.339954 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1da09-cef6-43c7-925b-edd8f224ce3f service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:27.340222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1da09-cef6-43c7-925b-edd8f224ce3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:27.340485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1da09-cef6-43c7-925b-edd8f224ce3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:27.342142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f5d1da09-cef6-43c7-925b-edd8f224ce3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:27.426151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:27.461123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1da09-cef6-43c7-925b-edd8f224ce3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:34:27.461123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1da09-cef6-43c7-925b-edd8f224ce3f service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:34:27.484564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 664/1331] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:34:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:34:30.475210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de39b1-3547-4013-bf32-4af76aeac779 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:30.475754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de39b1-3547-4013-bf32-4af76aeac779 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:30.475754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de39b1-3547-4013-bf32-4af76aeac779 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:30.477105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 668/1332] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:30.483174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1d431-18e3-47d3-9251-60b4d10f56e0 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:30.483579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1d431-18e3-47d3-9251-60b4d10f56e0 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:30.483719 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1d431-18e3-47d3-9251-60b4d10f56e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:30.549339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0ea1d431-18e3-47d3-9251-60b4d10f56e0 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:30.550552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0ea1d431-18e3-47d3-9251-60b4d10f56e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:30.651225 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 665/1333] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:34:30.935155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de39b1-3547-4013-bf32-4af76aeac779 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:34:30.936666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-46de39b1-3547-4013-bf32-4af76aeac779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:31.003900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:31.037066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46de39b1-3547-4013-bf32-4af76aeac779 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:34:31.111256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1f5fd-cc40-4ca6-aa17-45798e107ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:34:31.111858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f5fd-cc40-4ca6-aa17-45798e107ebd service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:31.112133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f5fd-cc40-4ca6-aa17-45798e107ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:31.112402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1f5fd-cc40-4ca6-aa17-45798e107ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:31.114704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f7e1f5fd-cc40-4ca6-aa17-45798e107ebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:31.220563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1f5fd-cc40-4ca6-aa17-45798e107ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:34:31.221940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1f5fd-cc40-4ca6-aa17-45798e107ebd service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:34:31.232212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:31.239968 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 669/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:34:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:34:33.499330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf008a-3efa-4ab8-9dc1-b059e6a08cca None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:33.499868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf008a-3efa-4ab8-9dc1-b059e6a08cca None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:33.499868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf008a-3efa-4ab8-9dc1-b059e6a08cca None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:33.501180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 666/1335] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:33.506386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01b48a-da6f-45db-be60-4a596d9a8e88 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:33.506769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01b48a-da6f-45db-be60-4a596d9a8e88 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:33.506920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01b48a-da6f-45db-be60-4a596d9a8e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:33.571655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-3c01b48a-da6f-45db-be60-4a596d9a8e88 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:33.573516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3c01b48a-da6f-45db-be60-4a596d9a8e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:33.668528 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 670/1336] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:34:33.959716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf008a-3efa-4ab8-9dc1-b059e6a08cca None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:34:33.961229 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6fdf008a-3efa-4ab8-9dc1-b059e6a08cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:34.001076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:34.065800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf008a-3efa-4ab8-9dc1-b059e6a08cca None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:34:34.137718 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c7a6a-e280-4ef7-bea4-242f2e593a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:34:34.138338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c7a6a-e280-4ef7-bea4-242f2e593a3d service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:34.138608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c7a6a-e280-4ef7-bea4-242f2e593a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:34.138871 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c7a6a-e280-4ef7-bea4-242f2e593a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:34.140418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d27c7a6a-e280-4ef7-bea4-242f2e593a3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:34.224600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:34.251133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c7a6a-e280-4ef7-bea4-242f2e593a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:34:34.252563 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c7a6a-e280-4ef7-bea4-242f2e593a3d service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:34:34.269960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 667/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:34:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:34:37.072917 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c09ab-9136-4177-85f1-1b98c9bdc31f None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:37.073589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c09ab-9136-4177-85f1-1b98c9bdc31f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:37.073589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c09ab-9136-4177-85f1-1b98c9bdc31f None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:37.074840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 671/1338] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:37.079972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42059053-b47b-4fc1-959b-8778fe132053 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:37.080425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42059053-b47b-4fc1-959b-8778fe132053 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:37.081039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42059053-b47b-4fc1-959b-8778fe132053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:37.144619 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-42059053-b47b-4fc1-959b-8778fe132053 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:37.145793 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-42059053-b47b-4fc1-959b-8778fe132053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:37.247409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 668/1339] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:34:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:34:37.530998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c09ab-9136-4177-85f1-1b98c9bdc31f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:34:37.532618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-900c09ab-9136-4177-85f1-1b98c9bdc31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:37.608164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:37.635046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c09ab-9136-4177-85f1-1b98c9bdc31f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:34:37.705605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0034dfd-a138-4d00-a75b-b15e06f77fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:34:37.706225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0034dfd-a138-4d00-a75b-b15e06f77fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:37.706497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0034dfd-a138-4d00-a75b-b15e06f77fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:37.706757 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0034dfd-a138-4d00-a75b-b15e06f77fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:37.708319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b0034dfd-a138-4d00-a75b-b15e06f77fb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:37.814083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0034dfd-a138-4d00-a75b-b15e06f77fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:34:37.815407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0034dfd-a138-4d00-a75b-b15e06f77fb1 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:34:37.832846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 672/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:34:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:34:42.297852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8e22e-3a2d-4f1b-b965-523a36018009 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:42.298425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8e22e-3a2d-4f1b-b965-523a36018009 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:42.298425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8e22e-3a2d-4f1b-b965-523a36018009 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:42.299943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 669/1341] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:42.305221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa344af-1395-4f50-84f3-1236abcb6b75 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:42.305646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa344af-1395-4f50-84f3-1236abcb6b75 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:42.307055 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa344af-1395-4f50-84f3-1236abcb6b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:42.312956 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:42.381182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-1aa344af-1395-4f50-84f3-1236abcb6b75 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:42.543926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-1aa344af-1395-4f50-84f3-1236abcb6b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:42.543926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 673/1342] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:34:42.775742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8e22e-3a2d-4f1b-b965-523a36018009 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:34:42.777324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1be8e22e-3a2d-4f1b-b965-523a36018009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:42.828598 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:42.880447 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8e22e-3a2d-4f1b-b965-523a36018009 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:34:42.958584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783e7fd-4a86-403a-adeb-1124df461848 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:34:42.959181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783e7fd-4a86-403a-adeb-1124df461848 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:42.959450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783e7fd-4a86-403a-adeb-1124df461848 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:42.959712 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783e7fd-4a86-403a-adeb-1124df461848 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:42.961273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7783e7fd-4a86-403a-adeb-1124df461848 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:43.057879 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:43.073707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783e7fd-4a86-403a-adeb-1124df461848 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:34:43.075321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783e7fd-4a86-403a-adeb-1124df461848 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:34:43.104334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 670/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:34:46.247316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03973c-0fe1-4a63-ab78-f882865e77b5 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:46.247870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03973c-0fe1-4a63-ab78-f882865e77b5 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:46.247870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03973c-0fe1-4a63-ab78-f882865e77b5 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:46.249366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 674/1344] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:46.254364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6309f-8a5f-4810-9c67-9e2a0ea3bedf None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:46.254822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6309f-8a5f-4810-9c67-9e2a0ea3bedf None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:46.254927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6309f-8a5f-4810-9c67-9e2a0ea3bedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:46.322251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-09e6309f-8a5f-4810-9c67-9e2a0ea3bedf None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:46.323449 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-09e6309f-8a5f-4810-9c67-9e2a0ea3bedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:46.424391 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 671/1345] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:34:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:34:46.713981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03973c-0fe1-4a63-ab78-f882865e77b5 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:34:46.715508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4d03973c-0fe1-4a63-ab78-f882865e77b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:46.805573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:46.822974 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03973c-0fe1-4a63-ab78-f882865e77b5 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:34:46.913414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335b9cf-9a61-4618-bac5-6de77497e8d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:34:46.913414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335b9cf-9a61-4618-bac5-6de77497e8d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:46.913414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335b9cf-9a61-4618-bac5-6de77497e8d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:46.913414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335b9cf-9a61-4618-bac5-6de77497e8d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:46.913917 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b335b9cf-9a61-4618-bac5-6de77497e8d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:47.034673 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335b9cf-9a61-4618-bac5-6de77497e8d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:34:47.036154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b335b9cf-9a61-4618-bac5-6de77497e8d4 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:34:47.056323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 675/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:34:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:34:51.802741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ba928-9883-4761-8206-5192a5470051 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:51.803281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ba928-9883-4761-8206-5192a5470051 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:51.803281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ba928-9883-4761-8206-5192a5470051 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:51.804603 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 672/1347] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:51.810829 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d1a1d-5df0-448c-af2d-848b5166a333 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:51.811241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d1a1d-5df0-448c-af2d-848b5166a333 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:51.811373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d1a1d-5df0-448c-af2d-848b5166a333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:51.826773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:51.879387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-0a3d1a1d-5df0-448c-af2d-848b5166a333 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:51.880511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0a3d1a1d-5df0-448c-af2d-848b5166a333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:51.981881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 676/1348] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:34:52.271826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ba928-9883-4761-8206-5192a5470051 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:34:52.273358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e17ba928-9883-4761-8206-5192a5470051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:52.332874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:52.378377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ba928-9883-4761-8206-5192a5470051 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:34:52.451497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc5d5f-e8e8-4a58-a8c4-1812da7630f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:34:52.452095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc5d5f-e8e8-4a58-a8c4-1812da7630f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:52.452367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc5d5f-e8e8-4a58-a8c4-1812da7630f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:52.452633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc5d5f-e8e8-4a58-a8c4-1812da7630f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:52.454237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b4fc5d5f-e8e8-4a58-a8c4-1812da7630f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:52.571494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:34:52.575251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc5d5f-e8e8-4a58-a8c4-1812da7630f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:34:52.576663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc5d5f-e8e8-4a58-a8c4-1812da7630f0 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:34:52.594827 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 673/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:34:57.668137 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbd25c-7374-4b1b-ace9-2b83e42cbd77 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:57.668691 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbd25c-7374-4b1b-ace9-2b83e42cbd77 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:57.668691 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbd25c-7374-4b1b-ace9-2b83e42cbd77 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:57.672373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 677/1350] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:34:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:34:57.677853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a00468-2385-4e0d-b3d7-49c602c3de44 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:34:57.678312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a00468-2385-4e0d-b3d7-49c602c3de44 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:34:57.678312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a00468-2385-4e0d-b3d7-49c602c3de44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:34:57.764490 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-b5a00468-2385-4e0d-b3d7-49c602c3de44 None None] MFA Rules not processed for user `9975c0dab76f4ccbb4d2107e8e780ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:34:57.765579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b5a00468-2385-4e0d-b3d7-49c602c3de44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:34:57.952785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 674/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:34:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:01.980267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=59 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:02.307176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec100a35-aee5-4e79-9683-7f8df372717b None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:02.307688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec100a35-aee5-4e79-9683-7f8df372717b None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:02.308191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec100a35-aee5-4e79-9683-7f8df372717b None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:02.309724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 678/1352] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:35:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:02.315054 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d816fc-70e9-45de-887d-2f55b053a957 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:02.315503 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d816fc-70e9-45de-887d-2f55b053a957 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:02.315586 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d816fc-70e9-45de-887d-2f55b053a957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:02.375326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:02.399589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-25d816fc-70e9-45de-887d-2f55b053a957 None None] MFA Rules not processed for user `9975c0dab76f4ccbb4d2107e8e780ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:02.400902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-25d816fc-70e9-45de-887d-2f55b053a957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:02.535947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 675/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:35:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:03.785006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f7253-0c3a-4d7e-ba85-160f9c783c96 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:03.785531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f7253-0c3a-4d7e-ba85-160f9c783c96 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:03.785531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f7253-0c3a-4d7e-ba85-160f9c783c96 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:03.786915 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 679/1354] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:35:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:03.792540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359d4fd-4681-4e82-819e-6b037b29e7b8 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:03.792987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359d4fd-4681-4e82-819e-6b037b29e7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:03.794596 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359d4fd-4681-4e82-819e-6b037b29e7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:03.877329 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c359d4fd-4681-4e82-819e-6b037b29e7b8 None None] MFA Rules not processed for user `9975c0dab76f4ccbb4d2107e8e780ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:03.878463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c359d4fd-4681-4e82-819e-6b037b29e7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:03.923763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:04.016886 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 676/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:35:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:07.748933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cd9c8-f23c-482c-adea-ce29af25fea4 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:07.749460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cd9c8-f23c-482c-adea-ce29af25fea4 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:07.749619 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cd9c8-f23c-482c-adea-ce29af25fea4 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:07.755088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 680/1356] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:07.756788 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e1527-d107-4c1e-8ab7-d74f15337276 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:07.757256 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e1527-d107-4c1e-8ab7-d74f15337276 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:07.757339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e1527-d107-4c1e-8ab7-d74f15337276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:07.829885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-1f6e1527-d107-4c1e-8ab7-d74f15337276 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:07.831010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1f6e1527-d107-4c1e-8ab7-d74f15337276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:07.941646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 677/1357] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:08.251065 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1827e5-c7f4-4dfe-a1b7-93f71876ec22 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:08.251350 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1827e5-c7f4-4dfe-a1b7-93f71876ec22 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:08.251623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1827e5-c7f4-4dfe-a1b7-93f71876ec22 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:08.253313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 681/1358] 10.222.0.49 () {54 vars in 962 bytes} [Fri Sep 16 05:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:08.262050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915fc9a-1e03-4259-ba6e-382e116d98f7 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:08.265897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915fc9a-1e03-4259-ba6e-382e116d98f7 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:08.265897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a915fc9a-1e03-4259-ba6e-382e116d98f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:08.284623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:08.347760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-a915fc9a-1e03-4259-ba6e-382e116d98f7 None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:08.349227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a915fc9a-1e03-4259-ba6e-382e116d98f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:08.470255 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 678/1359] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Sep 16 05:35:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:35:08.478810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1827e5-c7f4-4dfe-a1b7-93f71876ec22 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:08.480574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-dc1827e5-c7f4-4dfe-a1b7-93f71876ec22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:08.654239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1827e5-c7f4-4dfe-a1b7-93f71876ec22 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:08.731593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:08.772991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865f887-9ddf-446b-ae82-ef4874d6669f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:08.777303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865f887-9ddf-446b-ae82-ef4874d6669f service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:08.777451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865f887-9ddf-446b-ae82-ef4874d6669f service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:08.777650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865f887-9ddf-446b-ae82-ef4874d6669f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:08.779903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7865f887-9ddf-446b-ae82-ef4874d6669f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:08.877759 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865f887-9ddf-446b-ae82-ef4874d6669f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:08.879535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865f887-9ddf-446b-ae82-ef4874d6669f service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:08.900023 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 682/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:35:11.310962 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866cf39-6d91-4a27-9fec-c242e1658297 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:11.311494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866cf39-6d91-4a27-9fec-c242e1658297 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:11.311494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866cf39-6d91-4a27-9fec-c242e1658297 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:11.312828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 679/1361] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:11.318166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ed18d-c814-43a8-9c2d-9a62307e5697 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:11.318602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ed18d-c814-43a8-9c2d-9a62307e5697 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:11.318699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ed18d-c814-43a8-9c2d-9a62307e5697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:11.394372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-076ed18d-c814-43a8-9c2d-9a62307e5697 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:11.395539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-076ed18d-c814-43a8-9c2d-9a62307e5697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:11.434885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:11.513858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 683/1362] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:11.802692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866cf39-6d91-4a27-9fec-c242e1658297 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:11.804351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9866cf39-6d91-4a27-9fec-c242e1658297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:11.887371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:11.996520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866cf39-6d91-4a27-9fec-c242e1658297 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:12.140443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa95269-e61b-4435-9e31-c38783f7cb8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:12.140443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa95269-e61b-4435-9e31-c38783f7cb8a service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:12.140443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa95269-e61b-4435-9e31-c38783f7cb8a service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:12.140443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa95269-e61b-4435-9e31-c38783f7cb8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:12.142767 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9fa95269-e61b-4435-9e31-c38783f7cb8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:12.168394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:12.239196 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa95269-e61b-4435-9e31-c38783f7cb8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:12.240803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa95269-e61b-4435-9e31-c38783f7cb8a service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:12.260293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 680/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:35:14.940388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31014abb-c20a-4fda-ac28-48c7ab74651d None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:14.940976 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31014abb-c20a-4fda-ac28-48c7ab74651d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:14.940976 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31014abb-c20a-4fda-ac28-48c7ab74651d None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:14.942284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 684/1364] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:14.947818 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af2e4a-8a91-41e3-b0f2-e586d3e66281 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:14.948279 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af2e4a-8a91-41e3-b0f2-e586d3e66281 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:14.948371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af2e4a-8a91-41e3-b0f2-e586d3e66281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:15.017925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c5af2e4a-8a91-41e3-b0f2-e586d3e66281 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:15.019130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c5af2e4a-8a91-41e3-b0f2-e586d3e66281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:15.122460 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:15.130221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 681/1365] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:15.416222 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96ee80-7080-4b42-9865-4d7a80fc8a46 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:15.420534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96ee80-7080-4b42-9865-4d7a80fc8a46 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:15.420534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96ee80-7080-4b42-9865-4d7a80fc8a46 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:15.420534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 685/1366] 10.222.0.49 () {54 vars in 962 bytes} [Fri Sep 16 05:35:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:15.424910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d383f5-c05e-4964-9d24-8b779737488a None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:15.425253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d383f5-c05e-4964-9d24-8b779737488a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:15.425608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d383f5-c05e-4964-9d24-8b779737488a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:15.514526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-d3d383f5-c05e-4964-9d24-8b779737488a None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:15.515716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d3d383f5-c05e-4964-9d24-8b779737488a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:15.641332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 682/1367] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Sep 16 05:35:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:15.649840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96ee80-7080-4b42-9865-4d7a80fc8a46 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:15.651723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-eb96ee80-7080-4b42-9865-4d7a80fc8a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:15.807956 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96ee80-7080-4b42-9865-4d7a80fc8a46 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:15.828470 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:15.936981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5f81a-0b8b-407e-a543-7d9ddf56c42f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:15.937592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5f81a-0b8b-407e-a543-7d9ddf56c42f service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:15.937895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5f81a-0b8b-407e-a543-7d9ddf56c42f service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:15.938171 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5f81a-0b8b-407e-a543-7d9ddf56c42f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:15.939763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0ca5f81a-0b8b-407e-a543-7d9ddf56c42f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:16.038861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5f81a-0b8b-407e-a543-7d9ddf56c42f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:16.041154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5f81a-0b8b-407e-a543-7d9ddf56c42f service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:16.061195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 686/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:35:16.127980 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbb6a5-39bc-47e2-a475-a244cc5467c8 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:16.128540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbb6a5-39bc-47e2-a475-a244cc5467c8 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:16.128540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbb6a5-39bc-47e2-a475-a244cc5467c8 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:16.130089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 683/1369] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:16.136373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0762cc-47e5-4cd0-a2fa-aa0da23c6adb None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:16.136674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0762cc-47e5-4cd0-a2fa-aa0da23c6adb None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:16.136947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0762cc-47e5-4cd0-a2fa-aa0da23c6adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:16.169116 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:16.208159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-ec0762cc-47e5-4cd0-a2fa-aa0da23c6adb None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:16.209398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ec0762cc-47e5-4cd0-a2fa-aa0da23c6adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:16.323593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 687/1370] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:16.666827 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3487f-6c96-41a6-b72e-1e37e26d8bea None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:16.666827 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3487f-6c96-41a6-b72e-1e37e26d8bea None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:16.666827 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3487f-6c96-41a6-b72e-1e37e26d8bea None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:16.667384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 684/1371] 10.222.0.49 () {54 vars in 964 bytes} [Fri Sep 16 05:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:16.672146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea86e6c-b13a-4ca9-931d-e92daba8be29 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:16.672492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea86e6c-b13a-4ca9-931d-e92daba8be29 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:16.672829 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea86e6c-b13a-4ca9-931d-e92daba8be29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:16.755030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-7ea86e6c-b13a-4ca9-931d-e92daba8be29 None None] MFA Rules not processed for user `ff50e50587254bac912fb342a7e09891`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:16.756315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7ea86e6c-b13a-4ca9-931d-e92daba8be29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:16.807031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:16.842520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 688/1372] 10.222.0.49 () {58 vars in 1061 bytes} [Fri Sep 16 05:35:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:35:16.850078 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3487f-6c96-41a6-b72e-1e37e26d8bea None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:16.851637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-25a3487f-6c96-41a6-b72e-1e37e26d8bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:16.904615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:16.970857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3487f-6c96-41a6-b72e-1e37e26d8bea None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:17.049269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6dff6-fdfc-4a1d-b57b-27ae143931e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:17.049896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6dff6-fdfc-4a1d-b57b-27ae143931e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:17.050436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6dff6-fdfc-4a1d-b57b-27ae143931e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:17.050436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6dff6-fdfc-4a1d-b57b-27ae143931e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:17.051976 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b5b6dff6-fdfc-4a1d-b57b-27ae143931e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:17.159058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:17.173467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6dff6-fdfc-4a1d-b57b-27ae143931e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:17.175766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6dff6-fdfc-4a1d-b57b-27ae143931e8 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:17.201967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 685/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:35:18.512870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84373b3b-a118-4e10-a8d2-9cbc7c1afd57 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:18.513439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84373b3b-a118-4e10-a8d2-9cbc7c1afd57 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:18.513439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84373b3b-a118-4e10-a8d2-9cbc7c1afd57 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:18.514816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 689/1374] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:18.520221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d5e83-17e6-4ae4-aada-cdf3fbc10da2 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:18.520659 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d5e83-17e6-4ae4-aada-cdf3fbc10da2 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:18.520756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d5e83-17e6-4ae4-aada-cdf3fbc10da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:18.589009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-610d5e83-17e6-4ae4-aada-cdf3fbc10da2 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:18.590304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-610d5e83-17e6-4ae4-aada-cdf3fbc10da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:18.694123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 686/1375] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:18.998717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84373b3b-a118-4e10-a8d2-9cbc7c1afd57 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:19.000315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-84373b3b-a118-4e10-a8d2-9cbc7c1afd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:19.152212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84373b3b-a118-4e10-a8d2-9cbc7c1afd57 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:19.206654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:19.275826 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b8a02-525a-4fe5-9156-91c7b7762536 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:19.275826 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b8a02-525a-4fe5-9156-91c7b7762536 service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:19.275826 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b8a02-525a-4fe5-9156-91c7b7762536 service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:19.275826 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b8a02-525a-4fe5-9156-91c7b7762536 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:19.276957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-700b8a02-525a-4fe5-9156-91c7b7762536 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:19.370543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b8a02-525a-4fe5-9156-91c7b7762536 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:19.372127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b8a02-525a-4fe5-9156-91c7b7762536 service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:19.390866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 690/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:35:21.750139 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6bb6f-150b-44e6-a41d-3d5122a92e22 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:21.751080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6bb6f-150b-44e6-a41d-3d5122a92e22 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:21.751364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6bb6f-150b-44e6-a41d-3d5122a92e22 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:21.752795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 687/1377] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:21.758887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fc95f-e2c7-4b9b-a76b-fdb1d356ef30 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:21.759349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fc95f-e2c7-4b9b-a76b-fdb1d356ef30 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:21.759454 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fc95f-e2c7-4b9b-a76b-fdb1d356ef30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:21.825114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-e17fc95f-e2c7-4b9b-a76b-fdb1d356ef30 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:21.826323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e17fc95f-e2c7-4b9b-a76b-fdb1d356ef30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:21.833700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:21.937902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 691/1378] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:22.225231 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6bb6f-150b-44e6-a41d-3d5122a92e22 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:22.226765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2ca6bb6f-150b-44e6-a41d-3d5122a92e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:22.292678 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:22.382648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6bb6f-150b-44e6-a41d-3d5122a92e22 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:22.508937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa248d4-684c-4132-ad95-3d2ceb7b29e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:22.509631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa248d4-684c-4132-ad95-3d2ceb7b29e9 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:22.510015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa248d4-684c-4132-ad95-3d2ceb7b29e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:22.510353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa248d4-684c-4132-ad95-3d2ceb7b29e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:22.514450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4fa248d4-684c-4132-ad95-3d2ceb7b29e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:22.541024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:22.625760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa248d4-684c-4132-ad95-3d2ceb7b29e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:22.627480 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa248d4-684c-4132-ad95-3d2ceb7b29e9 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:22.649872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 688/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:35:25.015736 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fae8f-859b-40a2-a3c8-f3839a8f2c50 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:25.016168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fae8f-859b-40a2-a3c8-f3839a8f2c50 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:25.016477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fae8f-859b-40a2-a3c8-f3839a8f2c50 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:25.017734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 692/1380] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:25.023194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e504e-96dd-4077-be2c-afef1f5798da None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:25.023605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e504e-96dd-4077-be2c-afef1f5798da None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:25.023737 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e504e-96dd-4077-be2c-afef1f5798da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:25.094573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-406e504e-96dd-4077-be2c-afef1f5798da None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:25.095758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-406e504e-96dd-4077-be2c-afef1f5798da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:25.209045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:25.219128 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 689/1381] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:25.534349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374fae8f-859b-40a2-a3c8-f3839a8f2c50 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:25.535888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-374fae8f-859b-40a2-a3c8-f3839a8f2c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:25.684029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:25.694368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374fae8f-859b-40a2-a3c8-f3839a8f2c50 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:25.818611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd625d-384f-4cb8-a547-b9649756bad9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:25.819166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd625d-384f-4cb8-a547-b9649756bad9 service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:25.819439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd625d-384f-4cb8-a547-b9649756bad9 service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:25.819717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd625d-384f-4cb8-a547-b9649756bad9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:25.822001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-13dd625d-384f-4cb8-a547-b9649756bad9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:25.913435 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd625d-384f-4cb8-a547-b9649756bad9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:25.914987 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd625d-384f-4cb8-a547-b9649756bad9 service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:25.925758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:25.934702 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 693/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:35:28.837082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ebfb0-2d45-4096-8e51-09a190c54a26 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:28.837855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ebfb0-2d45-4096-8e51-09a190c54a26 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:28.837855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ebfb0-2d45-4096-8e51-09a190c54a26 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:28.838972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 690/1383] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:28.845698 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27125d7-8479-49f6-bd6d-8ba70d31cd6d None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:28.846247 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27125d7-8479-49f6-bd6d-8ba70d31cd6d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:28.846345 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27125d7-8479-49f6-bd6d-8ba70d31cd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:28.919615 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a27125d7-8479-49f6-bd6d-8ba70d31cd6d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:28.920870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a27125d7-8479-49f6-bd6d-8ba70d31cd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:29.037367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 694/1384] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:29.366717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ebfb0-2d45-4096-8e51-09a190c54a26 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:29.369048 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-619ebfb0-2d45-4096-8e51-09a190c54a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:29.539430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ebfb0-2d45-4096-8e51-09a190c54a26 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:29.623046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:29.672477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cab0c-07d3-41f7-a6e6-9a5b6cb8dae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:29.673128 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cab0c-07d3-41f7-a6e6-9a5b6cb8dae1 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:29.673522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cab0c-07d3-41f7-a6e6-9a5b6cb8dae1 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:29.673838 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cab0c-07d3-41f7-a6e6-9a5b6cb8dae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:29.675596 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-df4cab0c-07d3-41f7-a6e6-9a5b6cb8dae1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:29.772888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cab0c-07d3-41f7-a6e6-9a5b6cb8dae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:29.775270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cab0c-07d3-41f7-a6e6-9a5b6cb8dae1 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:29.805873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 691/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:35:32.008395 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c247a3-0f1a-4aa3-983b-f6914c633330 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:32.009001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c247a3-0f1a-4aa3-983b-f6914c633330 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:32.009001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c247a3-0f1a-4aa3-983b-f6914c633330 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:32.010294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 695/1386] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:32.015658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54033d1-f6ab-4930-a760-1cdfd19794e8 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:32.016139 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54033d1-f6ab-4930-a760-1cdfd19794e8 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:32.016248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54033d1-f6ab-4930-a760-1cdfd19794e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:32.083910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-b54033d1-f6ab-4930-a760-1cdfd19794e8 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:32.085135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b54033d1-f6ab-4930-a760-1cdfd19794e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:32.121464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:32.186469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 692/1387] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:32.472859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c247a3-0f1a-4aa3-983b-f6914c633330 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:32.474418 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b7c247a3-0f1a-4aa3-983b-f6914c633330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:32.552650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:32.624061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c247a3-0f1a-4aa3-983b-f6914c633330 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:32.739998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2b2aa-ed13-4934-b47d-389fca522dde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:32.740496 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2b2aa-ed13-4934-b47d-389fca522dde service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:32.740772 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2b2aa-ed13-4934-b47d-389fca522dde service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:32.741037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2b2aa-ed13-4934-b47d-389fca522dde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:32.742834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c6d2b2aa-ed13-4934-b47d-389fca522dde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:32.777297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d797e2-5e48-4575-8e72-2cad3335c915 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:32.777575 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d797e2-5e48-4575-8e72-2cad3335c915 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:32.777868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d797e2-5e48-4575-8e72-2cad3335c915 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:32.779179 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 693/1388] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:35:32.783539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:32.783967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841419e-be54-4565-8492-87953294ff98 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:32.784243 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841419e-be54-4565-8492-87953294ff98 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:32.784515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841419e-be54-4565-8492-87953294ff98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:32.840320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2b2aa-ed13-4934-b47d-389fca522dde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:32.842179 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2b2aa-ed13-4934-b47d-389fca522dde service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:32.857865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-6841419e-be54-4565-8492-87953294ff98 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:32.859232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6841419e-be54-4565-8492-87953294ff98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:32.860843 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 696/1389] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:35:32.958517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 694/1390] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:35:33.267940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7f635-28ba-45b7-bce7-ee656d828b7d None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:33.268420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7f635-28ba-45b7-bce7-ee656d828b7d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:33.268574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a7f635-28ba-45b7-bce7-ee656d828b7d None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:33.269794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 697/1391] 10.222.0.49 () {54 vars in 962 bytes} [Fri Sep 16 05:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:33.275930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42675e01-2498-46c5-a66e-8e3883c8bc16 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:33.276395 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42675e01-2498-46c5-a66e-8e3883c8bc16 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:33.276501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42675e01-2498-46c5-a66e-8e3883c8bc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:33.292721 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:33.355258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-42675e01-2498-46c5-a66e-8e3883c8bc16 None None] MFA Rules not processed for user `30d96ec6235843c5b67d87a3d0ff0f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:33.356458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-42675e01-2498-46c5-a66e-8e3883c8bc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:33.434710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 695/1392] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Sep 16 05:35:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:33.443774 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7f635-28ba-45b7-bce7-ee656d828b7d None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:33.445275 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-62a7f635-28ba-45b7-bce7-ee656d828b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:33.561972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a7f635-28ba-45b7-bce7-ee656d828b7d None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:33.624472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:33.643243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc310b-204e-4e94-b663-68df3273f51d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d96ec6235843c5b67d87a3d0ff0f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:33.644182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc310b-204e-4e94-b663-68df3273f51d service glance] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:33.644602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc310b-204e-4e94-b663-68df3273f51d service glance] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:33.644877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc310b-204e-4e94-b663-68df3273f51d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:33.646527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7afc310b-204e-4e94-b663-68df3273f51d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:33.759608 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc310b-204e-4e94-b663-68df3273f51d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:33.761381 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc310b-204e-4e94-b663-68df3273f51d service glance] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:33.785455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 698/1393] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:35:34.690623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97f464-e75d-4214-80ee-73b402404614 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:34.691201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97f464-e75d-4214-80ee-73b402404614 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:34.691201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97f464-e75d-4214-80ee-73b402404614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:34.759995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-be97f464-e75d-4214-80ee-73b402404614 None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:34.761204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-be97f464-e75d-4214-80ee-73b402404614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:34.823710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:34.856895 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 696/1394] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:35:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:34.904027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa4409-0f96-4ac4-a3e1-4438f2321660 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:34.904517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa4409-0f96-4ac4-a3e1-4438f2321660 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:34.904622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa4409-0f96-4ac4-a3e1-4438f2321660 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:34.905953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 699/1395] 10.222.0.49 () {54 vars in 943 bytes} [Fri Sep 16 05:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:34.912395 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d76dbe-c672-4af1-9d81-049ba83a715e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:34.912680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d76dbe-c672-4af1-9d81-049ba83a715e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:34.912949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d76dbe-c672-4af1-9d81-049ba83a715e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:34.992859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-60d76dbe-c672-4af1-9d81-049ba83a715e None None] MFA Rules not processed for user `b260fed0d99a4d3fa8f9c2ad79223219`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:34.994087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-60d76dbe-c672-4af1-9d81-049ba83a715e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:35.104967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 697/1396] 10.222.0.49 () {58 vars in 1040 bytes} [Fri Sep 16 05:35:34 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:35.114437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa4409-0f96-4ac4-a3e1-4438f2321660 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:35.116095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a1fa4409-0f96-4ac4-a3e1-4438f2321660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:35.209184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:35.241920 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016206f9-b4c2-4374-9794-522b739d396c None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:35.242202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016206f9-b4c2-4374-9794-522b739d396c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:35.242476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016206f9-b4c2-4374-9794-522b739d396c None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:35.244025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 698/1397] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 16 05:35:35.249770 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa81ae4-5991-4f57-9593-2c738458a717 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:35.250096 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa81ae4-5991-4f57-9593-2c738458a717 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:35.250366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa81ae4-5991-4f57-9593-2c738458a717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:35.260881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa4409-0f96-4ac4-a3e1-4438f2321660 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:35.290668 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:35.322416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-7aa81ae4-5991-4f57-9593-2c738458a717 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:35.323642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7aa81ae4-5991-4f57-9593-2c738458a717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:35.365920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314a791-966c-40ac-aa6a-c5f2fc5ee7c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:35.366536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314a791-966c-40ac-aa6a-c5f2fc5ee7c9 service swift] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:35.366813 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314a791-966c-40ac-aa6a-c5f2fc5ee7c9 service swift] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:35.367081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314a791-966c-40ac-aa6a-c5f2fc5ee7c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:35.368791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e314a791-966c-40ac-aa6a-c5f2fc5ee7c9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:35.424241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 699/1398] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:35:35.445934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:35.457679 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314a791-966c-40ac-aa6a-c5f2fc5ee7c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:35.459347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314a791-966c-40ac-aa6a-c5f2fc5ee7c9 service swift] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:35.467592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 700/1399] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:35:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:35:35.719909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa81ae4-5991-4f57-9593-2c738458a717 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:35.721137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7aa81ae4-5991-4f57-9593-2c738458a717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:35.856611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:35.859019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa81ae4-5991-4f57-9593-2c738458a717 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:35.988941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755343d-16ac-4530-8ab6-59d0dc9e5291 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:35.989591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755343d-16ac-4530-8ab6-59d0dc9e5291 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:35.989898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755343d-16ac-4530-8ab6-59d0dc9e5291 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:35.990286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755343d-16ac-4530-8ab6-59d0dc9e5291 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:35.992034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7755343d-16ac-4530-8ab6-59d0dc9e5291 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:36.091487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755343d-16ac-4530-8ab6-59d0dc9e5291 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:36.093088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755343d-16ac-4530-8ab6-59d0dc9e5291 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:36.113673 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 700/1400] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:35:38.474939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b4fb3-64ea-4197-a8ee-73920fe6ae18 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:38.475501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b4fb3-64ea-4197-a8ee-73920fe6ae18 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:38.475501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b4fb3-64ea-4197-a8ee-73920fe6ae18 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:38.476853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 701/1401] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:38.482325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94e62f-5900-40e8-8f20-38bba0bbbdc8 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:38.482754 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94e62f-5900-40e8-8f20-38bba0bbbdc8 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:38.482947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94e62f-5900-40e8-8f20-38bba0bbbdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:38.487423 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:38.565155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-9c94e62f-5900-40e8-8f20-38bba0bbbdc8 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:38.566416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9c94e62f-5900-40e8-8f20-38bba0bbbdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:38.696546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 701/1402] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:39.019798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b4fb3-64ea-4197-a8ee-73920fe6ae18 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:39.021555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-796b4fb3-64ea-4197-a8ee-73920fe6ae18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:39.198091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b4fb3-64ea-4197-a8ee-73920fe6ae18 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:39.284824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:39.338955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60124b6-b348-45e0-8599-202a3b4e01dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:39.339636 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60124b6-b348-45e0-8599-202a3b4e01dd service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:39.339913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60124b6-b348-45e0-8599-202a3b4e01dd service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:39.375978 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60124b6-b348-45e0-8599-202a3b4e01dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:39.375978 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e60124b6-b348-45e0-8599-202a3b4e01dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:39.460793 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60124b6-b348-45e0-8599-202a3b4e01dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:39.462421 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60124b6-b348-45e0-8599-202a3b4e01dd service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:39.491606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 702/1403] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:35:42.229269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41ec17-85e2-4702-a303-07cb37cc0049 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:42.229806 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41ec17-85e2-4702-a303-07cb37cc0049 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:42.229806 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41ec17-85e2-4702-a303-07cb37cc0049 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:42.231117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 702/1404] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:42.236407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bba643-5f4b-4725-ad74-5cf0092c7f66 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:42.236851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bba643-5f4b-4725-ad74-5cf0092c7f66 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:42.236946 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bba643-5f4b-4725-ad74-5cf0092c7f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:42.313811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-f7bba643-5f4b-4725-ad74-5cf0092c7f66 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:42.315049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f7bba643-5f4b-4725-ad74-5cf0092c7f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:42.346371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:42.428125 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 703/1405] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:35:42.726301 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41ec17-85e2-4702-a303-07cb37cc0049 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:42.728397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7a41ec17-85e2-4702-a303-07cb37cc0049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:42.825427 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:42.905702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41ec17-85e2-4702-a303-07cb37cc0049 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:43.038382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5897f3e-9c3f-45b6-bbec-b4e4663409f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:43.039002 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5897f3e-9c3f-45b6-bbec-b4e4663409f5 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:43.039976 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5897f3e-9c3f-45b6-bbec-b4e4663409f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:43.040489 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5897f3e-9c3f-45b6-bbec-b4e4663409f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:43.042251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e5897f3e-9c3f-45b6-bbec-b4e4663409f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:43.094869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:43.150971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5897f3e-9c3f-45b6-bbec-b4e4663409f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:43.152655 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5897f3e-9c3f-45b6-bbec-b4e4663409f5 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:43.175560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 703/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:35:45.650293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cbc11-96a9-47db-9623-c92dbd1b6a43 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:45.651163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cbc11-96a9-47db-9623-c92dbd1b6a43 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:45.651163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cbc11-96a9-47db-9623-c92dbd1b6a43 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:45.652434 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 704/1407] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:35:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:35:45.657905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5adf08-9f6c-4810-a111-bfaf2588c4b0 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:45.658382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5adf08-9f6c-4810-a111-bfaf2588c4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:45.658473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5adf08-9f6c-4810-a111-bfaf2588c4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:45.734547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-da5adf08-9f6c-4810-a111-bfaf2588c4b0 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:35:45.735892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-da5adf08-9f6c-4810-a111-bfaf2588c4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:45.848065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 704/1408] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:35:46.140885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cbc11-96a9-47db-9623-c92dbd1b6a43 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:35:46.142508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-bf7cbc11-96a9-47db-9623-c92dbd1b6a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:46.311742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cbc11-96a9-47db-9623-c92dbd1b6a43 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:35:46.321964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:46.437646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251c332-5b8b-4e1f-9908-72972156c758 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:35:46.438466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251c332-5b8b-4e1f-9908-72972156c758 service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:35:46.438736 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251c332-5b8b-4e1f-9908-72972156c758 service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:35:46.439008 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251c332-5b8b-4e1f-9908-72972156c758 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:35:46.440587 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4251c332-5b8b-4e1f-9908-72972156c758 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:35:46.541546 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251c332-5b8b-4e1f-9908-72972156c758 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:35:46.543267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251c332-5b8b-4e1f-9908-72972156c758 service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:35:46.570925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 705/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:35:55.846155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:35:56.560959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=94 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:00.029156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d435b1-0d11-4a6e-9c59-a68c89e182e6 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:00.029725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d435b1-0d11-4a6e-9c59-a68c89e182e6 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:00.029725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d435b1-0d11-4a6e-9c59-a68c89e182e6 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:00.031026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 705/1410] 10.222.0.49 () {54 vars in 918 bytes} [Fri Sep 16 05:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:00.035886 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d741910-b2cb-43d0-b9ae-25a19efe7748 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:00.036317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d741910-b2cb-43d0-b9ae-25a19efe7748 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:00.036466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d741910-b2cb-43d0-b9ae-25a19efe7748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:00.111265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-6d741910-b2cb-43d0-b9ae-25a19efe7748 None None] MFA Rules not processed for user `9975c0dab76f4ccbb4d2107e8e780ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:00.112447 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-6d741910-b2cb-43d0-b9ae-25a19efe7748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:00.216560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 706/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Sep 16 05:36:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:00.230826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1623b04-51c8-4cc6-b14b-ca05fa5b6be3 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:00.231103 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1623b04-51c8-4cc6-b14b-ca05fa5b6be3 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:00.231370 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1623b04-51c8-4cc6-b14b-ca05fa5b6be3 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:00.232686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 706/1412] 10.222.0.49 () {54 vars in 918 bytes} [Fri Sep 16 05:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:00.237182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cc1b7-588a-4b8b-ab35-d88857873ca3 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:00.237514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cc1b7-588a-4b8b-ab35-d88857873ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:00.246369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cc1b7-588a-4b8b-ab35-d88857873ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:00.322236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:00.325278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-644cc1b7-588a-4b8b-ab35-d88857873ca3 None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:00.326503 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-644cc1b7-588a-4b8b-ab35-d88857873ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:00.439790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 707/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Fri Sep 16 05:36:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:03.100690 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086fd8c-89aa-4cae-ac50-a7219608d48d None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:03.101255 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086fd8c-89aa-4cae-ac50-a7219608d48d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:03.101255 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086fd8c-89aa-4cae-ac50-a7219608d48d None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:03.102660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:03.108083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491930fb-bb3e-4334-b19e-932b03f7f7c6 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:03.108497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491930fb-bb3e-4334-b19e-932b03f7f7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:03.108649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491930fb-bb3e-4334-b19e-932b03f7f7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:03.174797 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-491930fb-bb3e-4334-b19e-932b03f7f7c6 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:03.175908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-491930fb-bb3e-4334-b19e-932b03f7f7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:03.230032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:03.277537 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:03.568578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086fd8c-89aa-4cae-ac50-a7219608d48d None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:03.570233 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d086fd8c-89aa-4cae-ac50-a7219608d48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:03.720326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086fd8c-89aa-4cae-ac50-a7219608d48d None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:03.792020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:03.832013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2e0a6-7dd1-4090-9ea3-e6635b54f1b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:03.832653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2e0a6-7dd1-4090-9ea3-e6635b54f1b9 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:03.832924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2e0a6-7dd1-4090-9ea3-e6635b54f1b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:03.833183 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2e0a6-7dd1-4090-9ea3-e6635b54f1b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:03.834928 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-dad2e0a6-7dd1-4090-9ea3-e6635b54f1b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:03.926360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad2e0a6-7dd1-4090-9ea3-e6635b54f1b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:03.927943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad2e0a6-7dd1-4090-9ea3-e6635b54f1b9 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:03.946499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 708/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:05.161726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd685f-06bd-48e3-b649-0ef2470f4a23 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:05.162315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd685f-06bd-48e3-b649-0ef2470f4a23 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:05.162419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd685f-06bd-48e3-b649-0ef2470f4a23 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:05.164360 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 05:36:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:05.171278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb20e8f-1bed-465c-8eb5-593cafccd49f None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:05.171843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb20e8f-1bed-465c-8eb5-593cafccd49f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:05.171959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb20e8f-1bed-465c-8eb5-593cafccd49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:05.264342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-7eb20e8f-1bed-465c-8eb5-593cafccd49f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:05.265763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7eb20e8f-1bed-465c-8eb5-593cafccd49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:05.296446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:05.379917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 05:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:05.576936 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd685f-06bd-48e3-b649-0ef2470f4a23 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:05.578415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b2cd685f-06bd-48e3-b649-0ef2470f4a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:05.747353 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd685f-06bd-48e3-b649-0ef2470f4a23 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:05.792147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:05.866807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21596292-4b36-487e-8237-6bd7877bb4c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:05.867447 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21596292-4b36-487e-8237-6bd7877bb4c8 service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:05.867721 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21596292-4b36-487e-8237-6bd7877bb4c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:05.867989 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21596292-4b36-487e-8237-6bd7877bb4c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:05.869593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-21596292-4b36-487e-8237-6bd7877bb4c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:05.961534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21596292-4b36-487e-8237-6bd7877bb4c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:05.963128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21596292-4b36-487e-8237-6bd7877bb4c8 service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:05.980902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 710/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:07.117532 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b352a-b2bd-4294-b440-2bed5c89c00b None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:07.118147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b352a-b2bd-4294-b440-2bed5c89c00b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:07.118147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b352a-b2bd-4294-b440-2bed5c89c00b None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:07.119461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 05:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:07.125699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b4bd8-1c0f-41ca-8ec9-9e4bbac6f317 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:07.126186 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b4bd8-1c0f-41ca-8ec9-9e4bbac6f317 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:07.126404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8b4bd8-1c0f-41ca-8ec9-9e4bbac6f317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:07.180997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:07.202050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-6f8b4bd8-1c0f-41ca-8ec9-9e4bbac6f317 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:07.203241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-6f8b4bd8-1c0f-41ca-8ec9-9e4bbac6f317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:07.307937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 05:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:07.464093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b352a-b2bd-4294-b440-2bed5c89c00b None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:07.465550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a71b352a-b2bd-4294-b440-2bed5c89c00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:07.611797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b352a-b2bd-4294-b440-2bed5c89c00b None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:07.623796 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:07.724620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1beaa-3b7a-4e15-b945-e12f16e579ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:07.725146 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1beaa-3b7a-4e15-b945-e12f16e579ec service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:07.725419 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1beaa-3b7a-4e15-b945-e12f16e579ec service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:07.725685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1beaa-3b7a-4e15-b945-e12f16e579ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:07.727566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-88c1beaa-3b7a-4e15-b945-e12f16e579ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:07.815561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1beaa-3b7a-4e15-b945-e12f16e579ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:07.817144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1beaa-3b7a-4e15-b945-e12f16e579ec service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:07.835533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 711/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:08.956397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e9644-b302-4bf6-9dbd-2c477158b808 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:08.956917 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e9644-b302-4bf6-9dbd-2c477158b808 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:08.956917 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e9644-b302-4bf6-9dbd-2c477158b808 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:08.958316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 05:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:08.964668 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce0ae0-3152-4508-9d1f-4364c6368f39 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:08.965099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce0ae0-3152-4508-9d1f-4364c6368f39 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:08.965210 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce0ae0-3152-4508-9d1f-4364c6368f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:08.987114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:09.045081 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c5ce0ae0-3152-4508-9d1f-4364c6368f39 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:09.046311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c5ce0ae0-3152-4508-9d1f-4364c6368f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:09.147302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 05:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:09.300114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e9644-b302-4bf6-9dbd-2c477158b808 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:09.301792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8b2e9644-b302-4bf6-9dbd-2c477158b808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:09.409207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:09.444430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e9644-b302-4bf6-9dbd-2c477158b808 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:09.556494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfbc50-63f6-4948-95b3-44a2f24f5d53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:09.556980 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfbc50-63f6-4948-95b3-44a2f24f5d53 service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:09.557258 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfbc50-63f6-4948-95b3-44a2f24f5d53 service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:09.557525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfbc50-63f6-4948-95b3-44a2f24f5d53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:09.559183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-04cfbc50-63f6-4948-95b3-44a2f24f5d53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:09.635918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:09.649913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfbc50-63f6-4948-95b3-44a2f24f5d53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:09.651766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfbc50-63f6-4948-95b3-44a2f24f5d53 service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:09.669601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 713/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:10.795595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda604f-a376-4ec1-9ae0-9da103f444b3 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:10.796142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda604f-a376-4ec1-9ae0-9da103f444b3 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:10.796142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda604f-a376-4ec1-9ae0-9da103f444b3 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:10.797531 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 05:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:10.804032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c5928-3014-4e19-8f63-52b5ccf8e7dc None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:10.804594 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c5928-3014-4e19-8f63-52b5ccf8e7dc None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:10.804878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c5928-3014-4e19-8f63-52b5ccf8e7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:10.879383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-e70c5928-3014-4e19-8f63-52b5ccf8e7dc None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:10.880584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e70c5928-3014-4e19-8f63-52b5ccf8e7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:10.980002 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 05:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:11.135789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda604f-a376-4ec1-9ae0-9da103f444b3 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:11.137300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-acda604f-a376-4ec1-9ae0-9da103f444b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:11.224151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:11.292281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda604f-a376-4ec1-9ae0-9da103f444b3 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:11.407121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c1e83-0858-4e17-b4ea-45776b887b7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:11.407626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c1e83-0858-4e17-b4ea-45776b887b7e service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:11.407895 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c1e83-0858-4e17-b4ea-45776b887b7e service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:11.408153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c1e83-0858-4e17-b4ea-45776b887b7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:11.409845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-303c1e83-0858-4e17-b4ea-45776b887b7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:11.457615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:11.498539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c1e83-0858-4e17-b4ea-45776b887b7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:11.500201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c1e83-0858-4e17-b4ea-45776b887b7e service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:11.518035 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 714/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:12.612757 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2853210-cb98-4239-8bb8-61ee8e6dc8b7 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:12.613321 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2853210-cb98-4239-8bb8-61ee8e6dc8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:12.613321 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2853210-cb98-4239-8bb8-61ee8e6dc8b7 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:12.614673 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 05:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:12.621519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619ff05-e7da-468b-a09d-a77b488c8886 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:12.622019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619ff05-e7da-468b-a09d-a77b488c8886 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:12.622127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619ff05-e7da-468b-a09d-a77b488c8886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:12.698802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f619ff05-e7da-468b-a09d-a77b488c8886 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:12.699980 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f619ff05-e7da-468b-a09d-a77b488c8886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:12.795267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 05:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:12.947433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2853210-cb98-4239-8bb8-61ee8e6dc8b7 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:12.948973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f2853210-cb98-4239-8bb8-61ee8e6dc8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:12.997596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:13.094807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2853210-cb98-4239-8bb8-61ee8e6dc8b7 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:13.214054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1367d272-3fc8-4a2b-810b-3cdbc249cc09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:13.214582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367d272-3fc8-4a2b-810b-3cdbc249cc09 service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:13.214853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367d272-3fc8-4a2b-810b-3cdbc249cc09 service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:13.215125 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1367d272-3fc8-4a2b-810b-3cdbc249cc09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:13.216762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-1367d272-3fc8-4a2b-810b-3cdbc249cc09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:13.223687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:13.313136 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367d272-3fc8-4a2b-810b-3cdbc249cc09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:13.314770 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1367d272-3fc8-4a2b-810b-3cdbc249cc09 service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:13.332617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 716/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:14.472158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d38d44-7c0e-4428-a54c-0d57cd7d7eea None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:14.472735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d38d44-7c0e-4428-a54c-0d57cd7d7eea None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:14.472735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d38d44-7c0e-4428-a54c-0d57cd7d7eea None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:14.474044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 05:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:14.481207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95857aa9-e2d2-4d07-9da4-c759ef7baa12 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:14.481682 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95857aa9-e2d2-4d07-9da4-c759ef7baa12 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:14.481782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95857aa9-e2d2-4d07-9da4-c759ef7baa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:14.556668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-95857aa9-e2d2-4d07-9da4-c759ef7baa12 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:14.557931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-95857aa9-e2d2-4d07-9da4-c759ef7baa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:14.624373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:14.653084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 05:36:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:14.804853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d38d44-7c0e-4428-a54c-0d57cd7d7eea None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:14.806342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-85d38d44-7c0e-4428-a54c-0d57cd7d7eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:14.824156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:14.952502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d38d44-7c0e-4428-a54c-0d57cd7d7eea None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:15.041745 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:15.067875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad1c89-66f6-4525-aedd-f9ae45dfb799 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:15.068435 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad1c89-66f6-4525-aedd-f9ae45dfb799 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:15.068704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad1c89-66f6-4525-aedd-f9ae45dfb799 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:15.068964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad1c89-66f6-4525-aedd-f9ae45dfb799 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:15.070877 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-31ad1c89-66f6-4525-aedd-f9ae45dfb799 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:15.164158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad1c89-66f6-4525-aedd-f9ae45dfb799 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:15.165736 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad1c89-66f6-4525-aedd-f9ae45dfb799 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:15.183829 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 717/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:16.302144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402fed7-dbf5-4ee5-b270-754d67b19542 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:16.302712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402fed7-dbf5-4ee5-b270-754d67b19542 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:16.302712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402fed7-dbf5-4ee5-b270-754d67b19542 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:16.304582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 05:36:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:16.311126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fa698-bc06-478e-8898-77c9cb3a6f31 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:16.311612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fa698-bc06-478e-8898-77c9cb3a6f31 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:16.311714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fa698-bc06-478e-8898-77c9cb3a6f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:16.386823 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-4e7fa698-bc06-478e-8898-77c9cb3a6f31 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:16.388021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4e7fa698-bc06-478e-8898-77c9cb3a6f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:16.430607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:16.487705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 05:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:16.636325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402fed7-dbf5-4ee5-b270-754d67b19542 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:16.637851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-9402fed7-dbf5-4ee5-b270-754d67b19542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:16.781636 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402fed7-dbf5-4ee5-b270-754d67b19542 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:16.834099 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:16.891199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c203fc-9320-41df-8f4c-2fccbe6a17d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:16.891717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c203fc-9320-41df-8f4c-2fccbe6a17d8 service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:16.891987 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c203fc-9320-41df-8f4c-2fccbe6a17d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:16.892250 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c203fc-9320-41df-8f4c-2fccbe6a17d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:16.893856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-86c203fc-9320-41df-8f4c-2fccbe6a17d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:16.982040 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c203fc-9320-41df-8f4c-2fccbe6a17d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:16.983666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c203fc-9320-41df-8f4c-2fccbe6a17d8 service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:17.001633 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 719/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:18.115247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3c1bf-6214-4f32-911f-2846341fc335 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:18.115824 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3c1bf-6214-4f32-911f-2846341fc335 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:18.115824 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3c1bf-6214-4f32-911f-2846341fc335 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:18.117187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 05:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:18.123378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2252c-2918-4341-b382-5d8d6e9a3c7c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:18.123834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2252c-2918-4341-b382-5d8d6e9a3c7c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:18.124004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2252c-2918-4341-b382-5d8d6e9a3c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:18.201536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-50e2252c-2918-4341-b382-5d8d6e9a3c7c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:18.202699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-50e2252c-2918-4341-b382-5d8d6e9a3c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:18.204705 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:18.298391 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 05:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:18.450200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3c1bf-6214-4f32-911f-2846341fc335 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:18.451904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-bdb3c1bf-6214-4f32-911f-2846341fc335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:18.713461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3c1bf-6214-4f32-911f-2846341fc335 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:18.739560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:18.825149 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8296eedb-f6a3-437c-bf23-215e43a84693 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:18.825747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296eedb-f6a3-437c-bf23-215e43a84693 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:18.826050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296eedb-f6a3-437c-bf23-215e43a84693 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:18.826314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296eedb-f6a3-437c-bf23-215e43a84693 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:18.827892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8296eedb-f6a3-437c-bf23-215e43a84693 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:18.916144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296eedb-f6a3-437c-bf23-215e43a84693 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:18.918168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8296eedb-f6a3-437c-bf23-215e43a84693 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:18.936149 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 720/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:21.076733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721ea9b-5346-46f6-b4eb-39801cbac245 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:21.077266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721ea9b-5346-46f6-b4eb-39801cbac245 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:21.077266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721ea9b-5346-46f6-b4eb-39801cbac245 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:21.078647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:21.084042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aafb94-5ce6-4fb1-9139-bde5eb1489f5 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:21.084454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aafb94-5ce6-4fb1-9139-bde5eb1489f5 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:21.084562 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aafb94-5ce6-4fb1-9139-bde5eb1489f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:21.126799 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:21.153907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c4aafb94-5ce6-4fb1-9139-bde5eb1489f5 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:21.155120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c4aafb94-5ce6-4fb1-9139-bde5eb1489f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:21.252130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:21.542753 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721ea9b-5346-46f6-b4eb-39801cbac245 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:21.544258 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e721ea9b-5346-46f6-b4eb-39801cbac245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:21.651495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721ea9b-5346-46f6-b4eb-39801cbac245 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:21.682908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:21.723472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665f24b-b133-4a26-99ee-c192eaabd290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:21.724071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665f24b-b133-4a26-99ee-c192eaabd290 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:21.724372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665f24b-b133-4a26-99ee-c192eaabd290 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:21.724688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665f24b-b133-4a26-99ee-c192eaabd290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:21.726563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-6665f24b-b133-4a26-99ee-c192eaabd290 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:21.831974 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665f24b-b133-4a26-99ee-c192eaabd290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:21.833309 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665f24b-b133-4a26-99ee-c192eaabd290 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:21.852296 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 722/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:24.214976 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981aa29-23a0-4978-a34d-053e72b9e950 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:24.215506 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981aa29-23a0-4978-a34d-053e72b9e950 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:24.215506 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981aa29-23a0-4978-a34d-053e72b9e950 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:24.216724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:24.221879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056d331-27e6-4fd0-9b1e-dd1fd462e2db None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:24.222262 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056d331-27e6-4fd0-9b1e-dd1fd462e2db None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:24.222443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056d331-27e6-4fd0-9b1e-dd1fd462e2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:24.287962 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-e056d331-27e6-4fd0-9b1e-dd1fd462e2db None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:24.289754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e056d331-27e6-4fd0-9b1e-dd1fd462e2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:24.302331 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:24.385193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:24.670907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981aa29-23a0-4978-a34d-053e72b9e950 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:24.672426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8981aa29-23a0-4978-a34d-053e72b9e950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:24.776734 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981aa29-23a0-4978-a34d-053e72b9e950 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:24.790209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:24.851708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af256f9-8cd4-4ded-a983-7db4217eca65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:24.852298 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af256f9-8cd4-4ded-a983-7db4217eca65 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:24.852571 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af256f9-8cd4-4ded-a983-7db4217eca65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:24.852837 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af256f9-8cd4-4ded-a983-7db4217eca65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:24.854450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6af256f9-8cd4-4ded-a983-7db4217eca65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:24.963803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af256f9-8cd4-4ded-a983-7db4217eca65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:24.965216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af256f9-8cd4-4ded-a983-7db4217eca65 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:24.983520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 723/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:27.603192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed553bc4-e68e-45ff-93f2-7f3bee3e9768 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:27.603768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed553bc4-e68e-45ff-93f2-7f3bee3e9768 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:27.603894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed553bc4-e68e-45ff-93f2-7f3bee3e9768 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:27.605199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:36:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:27.610690 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e0f5-34e3-4300-982c-595934435c0f None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:27.611147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e0f5-34e3-4300-982c-595934435c0f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:27.611253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e0f5-34e3-4300-982c-595934435c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:27.661560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:27.680045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-3c12e0f5-34e3-4300-982c-595934435c0f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:27.681252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3c12e0f5-34e3-4300-982c-595934435c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:27.778811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:28.076966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed553bc4-e68e-45ff-93f2-7f3bee3e9768 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:28.078547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ed553bc4-e68e-45ff-93f2-7f3bee3e9768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:28.184699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed553bc4-e68e-45ff-93f2-7f3bee3e9768 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:28.228513 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:28.261427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51da788-7eec-4608-b82b-a947b962ed83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:28.262062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51da788-7eec-4608-b82b-a947b962ed83 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:28.262333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51da788-7eec-4608-b82b-a947b962ed83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:28.262597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51da788-7eec-4608-b82b-a947b962ed83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:28.264170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c51da788-7eec-4608-b82b-a947b962ed83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:28.374404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51da788-7eec-4608-b82b-a947b962ed83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:28.375825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51da788-7eec-4608-b82b-a947b962ed83 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:28.394196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 725/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:30.706066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988833ce-959e-449c-ace7-1649c1c97f8f None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:30.706635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988833ce-959e-449c-ace7-1649c1c97f8f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:30.706635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988833ce-959e-449c-ace7-1649c1c97f8f None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:30.707969 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:30.713498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7d665-1838-4fcf-8421-1f735f80c7cf None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:30.713923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7d665-1838-4fcf-8421-1f735f80c7cf None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:30.714074 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7d665-1838-4fcf-8421-1f735f80c7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:30.779247 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-31c7d665-1838-4fcf-8421-1f735f80c7cf None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:30.780427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-31c7d665-1838-4fcf-8421-1f735f80c7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:30.801243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:30.877547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:31.167862 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988833ce-959e-449c-ace7-1649c1c97f8f None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:31.169362 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-988833ce-959e-449c-ace7-1649c1c97f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:31.273367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988833ce-959e-449c-ace7-1649c1c97f8f None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:31.294259 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:31.350225 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d21ee2-219f-4313-a33e-559d6ae849f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:31.351677 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21ee2-219f-4313-a33e-559d6ae849f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:31.352039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21ee2-219f-4313-a33e-559d6ae849f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:31.352309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21ee2-219f-4313-a33e-559d6ae849f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:31.353957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-93d21ee2-219f-4313-a33e-559d6ae849f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:31.462314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d21ee2-219f-4313-a33e-559d6ae849f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:31.463657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d21ee2-219f-4313-a33e-559d6ae849f2 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:31.481527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 726/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:33.706713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee0375-1896-41ab-8860-ee4123ca11b5 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:33.707294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee0375-1896-41ab-8860-ee4123ca11b5 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:33.707294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee0375-1896-41ab-8860-ee4123ca11b5 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:33.708601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:33.714074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2d3e0-6cdf-4cb7-9013-6eee96e8660f None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:33.714514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2d3e0-6cdf-4cb7-9013-6eee96e8660f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:33.714615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2d3e0-6cdf-4cb7-9013-6eee96e8660f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:33.779288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:33.781720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-dab2d3e0-6cdf-4cb7-9013-6eee96e8660f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:33.782944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-dab2d3e0-6cdf-4cb7-9013-6eee96e8660f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:33.879765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:34.163749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee0375-1896-41ab-8860-ee4123ca11b5 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:34.165285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5bee0375-1896-41ab-8860-ee4123ca11b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:34.268872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee0375-1896-41ab-8860-ee4123ca11b5 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:34.318784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:34.343472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6fc87-9197-4595-8caa-1336b291d03d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:34.344070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6fc87-9197-4595-8caa-1336b291d03d service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:34.344345 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6fc87-9197-4595-8caa-1336b291d03d service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:34.344613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6fc87-9197-4595-8caa-1336b291d03d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:34.346235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-65a6fc87-9197-4595-8caa-1336b291d03d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:34.454197 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6fc87-9197-4595-8caa-1336b291d03d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:34.455834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6fc87-9197-4595-8caa-1336b291d03d service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:34.473875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 728/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:36.737511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca138e-57aa-4e4e-9ad3-e777596b4078 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:36.738123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca138e-57aa-4e4e-9ad3-e777596b4078 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:36.738123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca138e-57aa-4e4e-9ad3-e777596b4078 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:36.739415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:36.745146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35fa98-6715-4b45-ae72-e994469425df None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:36.745612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35fa98-6715-4b45-ae72-e994469425df None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:36.745732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35fa98-6715-4b45-ae72-e994469425df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:36.810747 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-2b35fa98-6715-4b45-ae72-e994469425df None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:36.811933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2b35fa98-6715-4b45-ae72-e994469425df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:36.840908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:36.913658 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:37.195894 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca138e-57aa-4e4e-9ad3-e777596b4078 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:37.197405 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f3ca138e-57aa-4e4e-9ad3-e777596b4078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:37.318448 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca138e-57aa-4e4e-9ad3-e777596b4078 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:37.348494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:37.401441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f5a88-d584-480b-8d94-eb5f4f006455 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:37.402091 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f5a88-d584-480b-8d94-eb5f4f006455 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:37.402356 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f5a88-d584-480b-8d94-eb5f4f006455 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:37.402611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f5a88-d584-480b-8d94-eb5f4f006455 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:37.404174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-891f5a88-d584-480b-8d94-eb5f4f006455 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:37.514564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f5a88-d584-480b-8d94-eb5f4f006455 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:37.516004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f5a88-d584-480b-8d94-eb5f4f006455 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:37.534509 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 729/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:39.755128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4bdbf-f9c5-4d12-b791-5700e25fb5d4 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:39.755697 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4bdbf-f9c5-4d12-b791-5700e25fb5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:39.755697 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4bdbf-f9c5-4d12-b791-5700e25fb5d4 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:39.757021 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:39.762383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6e06a-e2d1-421b-b15a-54aedde71990 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:39.762793 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6e06a-e2d1-421b-b15a-54aedde71990 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:39.762952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6e06a-e2d1-421b-b15a-54aedde71990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:39.829503 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c7e6e06a-e2d1-421b-b15a-54aedde71990 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:39.830750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c7e6e06a-e2d1-421b-b15a-54aedde71990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:39.839060 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:39.926263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:40.207233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4bdbf-f9c5-4d12-b791-5700e25fb5d4 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:40.208741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-97e4bdbf-f9c5-4d12-b791-5700e25fb5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:40.313036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4bdbf-f9c5-4d12-b791-5700e25fb5d4 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:40.369079 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:40.388233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f74bdb-814b-44b1-a1ff-d2d2c0622d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:40.388835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f74bdb-814b-44b1-a1ff-d2d2c0622d43 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:40.389107 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f74bdb-814b-44b1-a1ff-d2d2c0622d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:40.389369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f74bdb-814b-44b1-a1ff-d2d2c0622d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:40.390966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-78f74bdb-814b-44b1-a1ff-d2d2c0622d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:40.499994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f74bdb-814b-44b1-a1ff-d2d2c0622d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:40.501346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f74bdb-814b-44b1-a1ff-d2d2c0622d43 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:40.520456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 731/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:42.776797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41a675-05ec-4057-addf-abf993fd26c4 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:42.777375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41a675-05ec-4057-addf-abf993fd26c4 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:42.777375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41a675-05ec-4057-addf-abf993fd26c4 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:42.778678 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:42.783924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd73bf0-6a0f-40db-b9b3-cb348867455d None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:42.784385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd73bf0-6a0f-40db-b9b3-cb348867455d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:42.784492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd73bf0-6a0f-40db-b9b3-cb348867455d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:42.849407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-0fd73bf0-6a0f-40db-b9b3-cb348867455d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:42.851211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0fd73bf0-6a0f-40db-b9b3-cb348867455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:42.890312 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:42.955693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:43.250406 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41a675-05ec-4057-addf-abf993fd26c4 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:43.251917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-ed41a675-05ec-4057-addf-abf993fd26c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:43.354790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41a675-05ec-4057-addf-abf993fd26c4 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:43.400577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:43.440651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac2adf-dc1e-44e1-980f-44dfd10745bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:43.441172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac2adf-dc1e-44e1-980f-44dfd10745bb service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:43.441443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac2adf-dc1e-44e1-980f-44dfd10745bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:43.441730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac2adf-dc1e-44e1-980f-44dfd10745bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:43.444595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f0ac2adf-dc1e-44e1-980f-44dfd10745bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:43.553080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac2adf-dc1e-44e1-980f-44dfd10745bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:43.554916 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac2adf-dc1e-44e1-980f-44dfd10745bb service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:43.573720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 732/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:45.803355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851f0db-2600-49aa-b908-212f3ea70ae6 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:45.803901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851f0db-2600-49aa-b908-212f3ea70ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:45.804173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851f0db-2600-49aa-b908-212f3ea70ae6 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:45.805530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:45.810794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d4cef-4ff3-4cb9-a656-b5a54f332038 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:45.811179 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d4cef-4ff3-4cb9-a656-b5a54f332038 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:45.811328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d4cef-4ff3-4cb9-a656-b5a54f332038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:45.878357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-700d4cef-4ff3-4cb9-a656-b5a54f332038 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:45.879549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-700d4cef-4ff3-4cb9-a656-b5a54f332038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:45.903956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:45.981643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:46.270857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851f0db-2600-49aa-b908-212f3ea70ae6 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:46.271718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b851f0db-2600-49aa-b908-212f3ea70ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:46.375462 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851f0db-2600-49aa-b908-212f3ea70ae6 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:46.442844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:46.449801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6965f77-64c7-4611-be18-37041a4ace76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:46.450428 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6965f77-64c7-4611-be18-37041a4ace76 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:46.456932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6965f77-64c7-4611-be18-37041a4ace76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:46.456932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6965f77-64c7-4611-be18-37041a4ace76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:46.456932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e6965f77-64c7-4611-be18-37041a4ace76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:46.563245 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6965f77-64c7-4611-be18-37041a4ace76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:46.564572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6965f77-64c7-4611-be18-37041a4ace76 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:46.582191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 734/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:36:48.755366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580ada1-b807-4c0d-b416-d27302e6e667 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:48.755939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580ada1-b807-4c0d-b416-d27302e6e667 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:48.755939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580ada1-b807-4c0d-b416-d27302e6e667 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:48.757388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:36:48.762752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2182d-7366-4bd8-a9a1-29dfe835f41f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:48.763212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2182d-7366-4bd8-a9a1-29dfe835f41f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:48.763327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2182d-7366-4bd8-a9a1-29dfe835f41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:48.827469 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-4dd2182d-7366-4bd8-a9a1-29dfe835f41f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:36:48.828653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4dd2182d-7366-4bd8-a9a1-29dfe835f41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:48.875604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:48.924039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:36:49.209446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580ada1-b807-4c0d-b416-d27302e6e667 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:36:49.211442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5580ada1-b807-4c0d-b416-d27302e6e667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:49.317744 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580ada1-b807-4c0d-b416-d27302e6e667 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:36:49.369073 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:49.396101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0c7f9-0f17-488f-a177-e26138643104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:36:49.397012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0c7f9-0f17-488f-a177-e26138643104 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:36:49.397688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0c7f9-0f17-488f-a177-e26138643104 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:36:49.397688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0c7f9-0f17-488f-a177-e26138643104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:36:49.399659 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-63a0c7f9-0f17-488f-a177-e26138643104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:36:49.512952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0c7f9-0f17-488f-a177-e26138643104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:36:49.514332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0c7f9-0f17-488f-a177-e26138643104 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:36:49.534605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 735/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:36:58.922774 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=22 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:36:59.532205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=66 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:09.926768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af242eb-4b34-4de0-abfe-16f5a88cd894 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:09.927193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af242eb-4b34-4de0-abfe-16f5a88cd894 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:09.927312 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af242eb-4b34-4de0-abfe-16f5a88cd894 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:09.929497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 736/1471] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:37:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:37:09.934679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e0431-24c5-4a4e-b66a-5a9a12c9e055 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:09.935103 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e0431-24c5-4a4e-b66a-5a9a12c9e055 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:09.935292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e0431-24c5-4a4e-b66a-5a9a12c9e055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:10.001528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-8b2e0431-24c5-4a4e-b66a-5a9a12c9e055 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:37:10.002760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8b2e0431-24c5-4a4e-b66a-5a9a12c9e055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:10.097933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 736/1472] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:37:10.398608 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d083028-7f3c-4d24-af9e-b61d8bde59ad None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:10.398889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d083028-7f3c-4d24-af9e-b61d8bde59ad None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:10.399157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d083028-7f3c-4d24-af9e-b61d8bde59ad None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:10.400486 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 737/1473] 10.222.0.49 () {54 vars in 962 bytes} [Fri Sep 16 05:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:37:10.405566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48233a5-f552-4990-8c3a-a613c9c54abe None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:10.405864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48233a5-f552-4990-8c3a-a613c9c54abe None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:10.406134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48233a5-f552-4990-8c3a-a613c9c54abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:10.484353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f48233a5-f552-4990-8c3a-a613c9c54abe None None] MFA Rules not processed for user `30d96ec6235843c5b67d87a3d0ff0f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:37:10.485606 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f48233a5-f552-4990-8c3a-a613c9c54abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:10.507491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:10.563050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 737/1474] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Sep 16 05:37:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:37:10.571645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d083028-7f3c-4d24-af9e-b61d8bde59ad None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:37:10.573152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8d083028-7f3c-4d24-af9e-b61d8bde59ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:10.673668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d083028-7f3c-4d24-af9e-b61d8bde59ad None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:37:10.745872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6171f-0312-44e6-9ef1-aa23fcfac322 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d96ec6235843c5b67d87a3d0ff0f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:37:10.746466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6171f-0312-44e6-9ef1-aa23fcfac322 service glance] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:10.746737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6171f-0312-44e6-9ef1-aa23fcfac322 service glance] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:10.747001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6171f-0312-44e6-9ef1-aa23fcfac322 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:10.748566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ebe6171f-0312-44e6-9ef1-aa23fcfac322 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:10.803726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:10.858295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6171f-0312-44e6-9ef1-aa23fcfac322 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:37:10.859713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6171f-0312-44e6-9ef1-aa23fcfac322 service glance] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:37:10.878550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 738/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:37:13.170679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbfab8-4a6a-437b-a2f1-d3488af779ea None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:13.171238 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbfab8-4a6a-437b-a2f1-d3488af779ea None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:13.171238 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cbfab8-4a6a-437b-a2f1-d3488af779ea None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:13.172599 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 738/1476] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:37:13.178333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba6f8b-5aa4-4e94-af80-2af6f7ee6ebb None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:13.178768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba6f8b-5aa4-4e94-af80-2af6f7ee6ebb None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:13.178964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba6f8b-5aa4-4e94-af80-2af6f7ee6ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:13.246653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-99ba6f8b-5aa4-4e94-af80-2af6f7ee6ebb None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:37:13.247846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-99ba6f8b-5aa4-4e94-af80-2af6f7ee6ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:13.348886 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 739/1477] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:37:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:37:13.622994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbfab8-4a6a-437b-a2f1-d3488af779ea None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:37:13.624548 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c0cbfab8-4a6a-437b-a2f1-d3488af779ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:13.748908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cbfab8-4a6a-437b-a2f1-d3488af779ea None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:37:13.797769 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:13.847785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f26e6f-6e59-4eb0-b8f6-7d8a54d27c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:37:13.848382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f26e6f-6e59-4eb0-b8f6-7d8a54d27c54 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:13.848652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f26e6f-6e59-4eb0-b8f6-7d8a54d27c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:13.848908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f26e6f-6e59-4eb0-b8f6-7d8a54d27c54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:13.849591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f26e6f-6e59-4eb0-b8f6-7d8a54d27c54 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:37:13.851602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: WARNING py.warnings [None req-a1f26e6f-6e59-4eb0-b8f6-7d8a54d27c54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:37:13.851602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: warnings.warn(msg) Sep 16 05:37:13.851602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: Sep 16 05:37:13.852722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f26e6f-6e59-4eb0-b8f6-7d8a54d27c54 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:37:13.872007 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 739/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Fri Sep 16 05:37:13 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:37:16.072685 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191f693-eb5f-4fdc-86fa-d6958cc4e3b9 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:16.073239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191f693-eb5f-4fdc-86fa-d6958cc4e3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:16.073239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191f693-eb5f-4fdc-86fa-d6958cc4e3b9 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:16.074601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 740/1479] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:37:16.079941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de3bc3-af1d-4067-975d-04ce9a80b0af None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:16.080380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de3bc3-af1d-4067-975d-04ce9a80b0af None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:16.080491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de3bc3-af1d-4067-975d-04ce9a80b0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:16.147339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-10de3bc3-af1d-4067-975d-04ce9a80b0af None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:37:16.148552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-10de3bc3-af1d-4067-975d-04ce9a80b0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:16.253731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 740/1480] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:37:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:37:16.547979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191f693-eb5f-4fdc-86fa-d6958cc4e3b9 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:37:16.549465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f191f693-eb5f-4fdc-86fa-d6958cc4e3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:16.559086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:16.694126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191f693-eb5f-4fdc-86fa-d6958cc4e3b9 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:37:16.772047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:16.805542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d05745-9f6e-44bb-ba86-4e3519efdc2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:37:16.806093 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d05745-9f6e-44bb-ba86-4e3519efdc2d service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:16.806372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d05745-9f6e-44bb-ba86-4e3519efdc2d service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:16.806641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d05745-9f6e-44bb-ba86-4e3519efdc2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:16.808231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a9d05745-9f6e-44bb-ba86-4e3519efdc2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:16.897878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d05745-9f6e-44bb-ba86-4e3519efdc2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:37:16.899454 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d05745-9f6e-44bb-ba86-4e3519efdc2d service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:37:16.917663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 741/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:37:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:37:19.256517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84d556-b633-4bb7-8359-bd29f209c6bb None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:19.257101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84d556-b633-4bb7-8359-bd29f209c6bb None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:19.257101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d84d556-b633-4bb7-8359-bd29f209c6bb None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:19.258449 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 741/1482] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:37:19.263952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee771cc-dd8d-4a2e-88be-5f9c63ac3b29 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:19.264360 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee771cc-dd8d-4a2e-88be-5f9c63ac3b29 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:19.264520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee771cc-dd8d-4a2e-88be-5f9c63ac3b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:19.331193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-2ee771cc-dd8d-4a2e-88be-5f9c63ac3b29 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:37:19.332390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2ee771cc-dd8d-4a2e-88be-5f9c63ac3b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:19.379699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:19.432291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 742/1483] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:37:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:37:19.725155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84d556-b633-4bb7-8359-bd29f209c6bb None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:37:19.726837 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4d84d556-b633-4bb7-8359-bd29f209c6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:19.750706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:19.830680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d84d556-b633-4bb7-8359-bd29f209c6bb None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:37:19.905504 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13277059-a2d0-4502-9c7f-23b37892f0f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:37:19.906125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13277059-a2d0-4502-9c7f-23b37892f0f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:19.906398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13277059-a2d0-4502-9c7f-23b37892f0f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:19.906661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13277059-a2d0-4502-9c7f-23b37892f0f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:19.908221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-13277059-a2d0-4502-9c7f-23b37892f0f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:19.982041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:20.017045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13277059-a2d0-4502-9c7f-23b37892f0f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:37:20.018497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13277059-a2d0-4502-9c7f-23b37892f0f1 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:37:20.036186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 742/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:37:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:37:22.264014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96be55b-fefa-431b-9f0a-5d3c6c781365 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:22.264559 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96be55b-fefa-431b-9f0a-5d3c6c781365 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:22.264559 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96be55b-fefa-431b-9f0a-5d3c6c781365 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:22.265788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 743/1485] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:37:22.271233 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adb1d9-559d-438f-980f-3d93b42f12d6 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:22.271860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adb1d9-559d-438f-980f-3d93b42f12d6 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:22.271860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11adb1d9-559d-438f-980f-3d93b42f12d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:22.337808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-11adb1d9-559d-438f-980f-3d93b42f12d6 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:37:22.339027 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-11adb1d9-559d-438f-980f-3d93b42f12d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:22.433903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 743/1486] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:37:22.726022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96be55b-fefa-431b-9f0a-5d3c6c781365 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:37:22.727555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e96be55b-fefa-431b-9f0a-5d3c6c781365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:22.831462 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96be55b-fefa-431b-9f0a-5d3c6c781365 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:37:22.903662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:22.905100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162ea39-e543-46ea-bc81-f4a7882e9db6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:37:22.905697 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162ea39-e543-46ea-bc81-f4a7882e9db6 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:22.906005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162ea39-e543-46ea-bc81-f4a7882e9db6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:22.906275 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162ea39-e543-46ea-bc81-f4a7882e9db6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:22.907846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2162ea39-e543-46ea-bc81-f4a7882e9db6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:23.018903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162ea39-e543-46ea-bc81-f4a7882e9db6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:37:23.020262 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162ea39-e543-46ea-bc81-f4a7882e9db6 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:37:23.038539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 744/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:37:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:37:25.435098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782daa-dada-4be7-96b5-1bc99d6aa099 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:25.435654 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782daa-dada-4be7-96b5-1bc99d6aa099 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:25.435654 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782daa-dada-4be7-96b5-1bc99d6aa099 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:25.436992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 744/1488] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:37:25.442316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c885b6-b8e8-4d02-a0d4-ad19a1ba304f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:25.442764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c885b6-b8e8-4d02-a0d4-ad19a1ba304f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:25.442878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c885b6-b8e8-4d02-a0d4-ad19a1ba304f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:25.515247 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-b2c885b6-b8e8-4d02-a0d4-ad19a1ba304f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:37:25.516443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b2c885b6-b8e8-4d02-a0d4-ad19a1ba304f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:25.569712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:25.614633 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 745/1489] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:37:25.905690 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782daa-dada-4be7-96b5-1bc99d6aa099 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:37:25.907229 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2c782daa-dada-4be7-96b5-1bc99d6aa099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:25.943968 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:26.010166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782daa-dada-4be7-96b5-1bc99d6aa099 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:37:26.084239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d7ed1-262f-4a73-b14d-b56339d8d7dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:37:26.084819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d7ed1-262f-4a73-b14d-b56339d8d7dd service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:26.085093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d7ed1-262f-4a73-b14d-b56339d8d7dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:26.085350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d7ed1-262f-4a73-b14d-b56339d8d7dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:26.086962 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-051d7ed1-262f-4a73-b14d-b56339d8d7dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:26.165762 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:26.198605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d7ed1-262f-4a73-b14d-b56339d8d7dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:37:26.200021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d7ed1-262f-4a73-b14d-b56339d8d7dd service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:37:26.218554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 745/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:37:28.722856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ab191-b678-4e97-990a-d3e1f84ff6f4 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:28.723423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ab191-b678-4e97-990a-d3e1f84ff6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:28.723423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ab191-b678-4e97-990a-d3e1f84ff6f4 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:28.724829 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 746/1491] 10.222.0.49 () {54 vars in 927 bytes} [Fri Sep 16 05:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:37:28.730211 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3026f3-a013-4528-9fac-6e67700e81cf None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:28.730635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3026f3-a013-4528-9fac-6e67700e81cf None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:28.730738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3026f3-a013-4528-9fac-6e67700e81cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:28.797647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-4f3026f3-a013-4528-9fac-6e67700e81cf None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:37:28.798852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4f3026f3-a013-4528-9fac-6e67700e81cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:28.900661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 746/1492] 10.222.0.49 () {58 vars in 1024 bytes} [Fri Sep 16 05:37:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:37:29.193730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ab191-b678-4e97-990a-d3e1f84ff6f4 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:37:29.195375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8c5ab191-b678-4e97-990a-d3e1f84ff6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:29.298660 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ab191-b678-4e97-990a-d3e1f84ff6f4 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:37:29.377878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a65ada-d622-4af1-99dd-2734e1648204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:37:29.378492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a65ada-d622-4af1-99dd-2734e1648204 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:37:29.378768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a65ada-d622-4af1-99dd-2734e1648204 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:37:29.379037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a65ada-d622-4af1-99dd-2734e1648204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:37:29.381384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e3a65ada-d622-4af1-99dd-2734e1648204 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:37:29.391227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:29.506972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a65ada-d622-4af1-99dd-2734e1648204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:37:29.508379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a65ada-d622-4af1-99dd-2734e1648204 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:37:29.527835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 747/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:37:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:37:38.899162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=82 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:37:39.525222 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=50 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:39:13.987443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5956c-7f99-4228-827f-071918fab16e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:13.988018 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5956c-7f99-4228-827f-071918fab16e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:13.988018 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5956c-7f99-4228-827f-071918fab16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:14.069592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-9ac5956c-7f99-4228-827f-071918fab16e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:39:14.070825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9ac5956c-7f99-4228-827f-071918fab16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:39:14.178305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 747/1494] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:39:14.186070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4550c51-df31-447a-81c9-f11f65a6d65f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:14.186572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4550c51-df31-447a-81c9-f11f65a6d65f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:14.186680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4550c51-df31-447a-81c9-f11f65a6d65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:14.206689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-f4550c51-df31-447a-81c9-f11f65a6d65f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:39:14.207879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f4550c51-df31-447a-81c9-f11f65a6d65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:39:14.216924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 748/1495] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:39:14.221355 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5956c-7f99-4228-827f-071918fab16e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:39:14.230911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5956c-7f99-4228-827f-071918fab16e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:39:14.235321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb6336-8494-465e-8e86-87c12b55d6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:39:14.235904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb6336-8494-465e-8e86-87c12b55d6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:14.236172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb6336-8494-465e-8e86-87c12b55d6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:14.236430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb6336-8494-465e-8e86-87c12b55d6e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:14.237056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb6336-8494-465e-8e86-87c12b55d6e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:39:14.237345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb6336-8494-465e-8e86-87c12b55d6e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:39:14.238366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: WARNING py.warnings [None req-cecb6336-8494-465e-8e86-87c12b55d6e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:39:14.238366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: warnings.warn(msg) Sep 16 05:39:14.238366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: Sep 16 05:39:14.239236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb6336-8494-465e-8e86-87c12b55d6e6 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:39:14.250345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 748/1496] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:39:14.259556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4550c51-df31-447a-81c9-f11f65a6d65f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:39:14.268377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4550c51-df31-447a-81c9-f11f65a6d65f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:39:14.273474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acb2f6-6829-4749-acbe-e7a575fe48d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:39:14.274134 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acb2f6-6829-4749-acbe-e7a575fe48d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:14.274403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acb2f6-6829-4749-acbe-e7a575fe48d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:14.274666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acb2f6-6829-4749-acbe-e7a575fe48d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:14.275287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acb2f6-6829-4749-acbe-e7a575fe48d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:39:14.277288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: WARNING py.warnings [None req-c6acb2f6-6829-4749-acbe-e7a575fe48d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:39:14.277288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: warnings.warn(msg) Sep 16 05:39:14.277288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: Sep 16 05:39:14.278511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acb2f6-6829-4749-acbe-e7a575fe48d0 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:39:14.390848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 749/1497] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:39:14 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:39:14.394971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb6336-8494-465e-8e86-87c12b55d6e6 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:39:14.405205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb6336-8494-465e-8e86-87c12b55d6e6 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:39:14.453795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:39:14.460449 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48c493-4365-4624-8df2-2726ce23a1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:39:14.461074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48c493-4365-4624-8df2-2726ce23a1ab admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:14.466498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48c493-4365-4624-8df2-2726ce23a1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:14.466498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48c493-4365-4624-8df2-2726ce23a1ab admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:14.466498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48c493-4365-4624-8df2-2726ce23a1ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:39:14.466498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: WARNING py.warnings [None req-6f48c493-4365-4624-8df2-2726ce23a1ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:39:14.466498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: warnings.warn(msg) Sep 16 05:39:14.466498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: Sep 16 05:39:14.466498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48c493-4365-4624-8df2-2726ce23a1ab admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:39:14.569475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 749/1498] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:39:14 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:39:14.573856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acb2f6-6829-4749-acbe-e7a575fe48d0 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:39:14.585208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acb2f6-6829-4749-acbe-e7a575fe48d0 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:39:14.589812 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda45a4b-2531-43ab-b1ea-0d405ed02604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:39:14.592690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda45a4b-2531-43ab-b1ea-0d405ed02604 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:14.592690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda45a4b-2531-43ab-b1ea-0d405ed02604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:14.592690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda45a4b-2531-43ab-b1ea-0d405ed02604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:14.592690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda45a4b-2531-43ab-b1ea-0d405ed02604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:39:14.592690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: WARNING py.warnings [None req-cda45a4b-2531-43ab-b1ea-0d405ed02604 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:39:14.592690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: warnings.warn(msg) Sep 16 05:39:14.592690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: Sep 16 05:39:14.593594 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda45a4b-2531-43ab-b1ea-0d405ed02604 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:39:14.606783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 750/1499] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:39:14.610871 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48c493-4365-4624-8df2-2726ce23a1ab admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:39:14.621287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48c493-4365-4624-8df2-2726ce23a1ab admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:39:14.627411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eae292-c94e-4717-a641-6d0818d64750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:39:14.627411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eae292-c94e-4717-a641-6d0818d64750 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:14.627411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eae292-c94e-4717-a641-6d0818d64750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:14.627411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eae292-c94e-4717-a641-6d0818d64750 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:14.627787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eae292-c94e-4717-a641-6d0818d64750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:39:14.628582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: WARNING py.warnings [None req-24eae292-c94e-4717-a641-6d0818d64750 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:39:14.628582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: warnings.warn(msg) Sep 16 05:39:14.628582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: Sep 16 05:39:14.629448 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eae292-c94e-4717-a641-6d0818d64750 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:39:14.643490 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 750/1500] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:39:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:39:14.648153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda45a4b-2531-43ab-b1ea-0d405ed02604 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:39:14.658492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda45a4b-2531-43ab-b1ea-0d405ed02604 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:39:14.663435 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dce90-ff1b-4f54-9372-be8018a99567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:39:14.664084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dce90-ff1b-4f54-9372-be8018a99567 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:14.664359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dce90-ff1b-4f54-9372-be8018a99567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:14.664631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dce90-ff1b-4f54-9372-be8018a99567 admin admin] PATH_INFO: `/v3/projects/55d37540ac824e8d87afd8b8615ee1b4/users/cb80e7671eb24d58977bfc432249aee4/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:14.679741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dce90-ff1b-4f54-9372-be8018a99567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55d37540ac824e8d87afd8b8615ee1b4, user_id=cb80e7671eb24d58977bfc432249aee4, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:39:14.680792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: WARNING py.warnings [None req-9a3dce90-ff1b-4f54-9372-be8018a99567 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:39:14.680792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: warnings.warn(msg) Sep 16 05:39:14.680792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: Sep 16 05:39:14.682749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dce90-ff1b-4f54-9372-be8018a99567 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:39:14.778241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 751/1501] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:39:14 2022] PUT /identity/v3/projects/55d37540ac824e8d87afd8b8615ee1b4/users/cb80e7671eb24d58977bfc432249aee4/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:39:14.791042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a0d16-ce56-4067-addc-a3519f6bf55b None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:14.791341 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a0d16-ce56-4067-addc-a3519f6bf55b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:14.791630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a0d16-ce56-4067-addc-a3519f6bf55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:14.818908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-485a0d16-ce56-4067-addc-a3519f6bf55b None None] MFA Rules not processed for user `cb80e7671eb24d58977bfc432249aee4`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:39:14.820110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-485a0d16-ce56-4067-addc-a3519f6bf55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:39:14.882775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 751/1502] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:39:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:39:14.896943 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d988bb-680a-4a68-8df2-77ab04c34ecd None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:14.897234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d988bb-680a-4a68-8df2-77ab04c34ecd None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:14.897503 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d988bb-680a-4a68-8df2-77ab04c34ecd None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:14.898892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 752/1503] 10.222.0.49 () {54 vars in 964 bytes} [Fri Sep 16 05:39:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:39:14.903808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec3ea8-631d-4fe4-8c97-18b1bd6edf8f None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:14.904089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec3ea8-631d-4fe4-8c97-18b1bd6edf8f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:14.904358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec3ea8-631d-4fe4-8c97-18b1bd6edf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:14.962946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-e3ec3ea8-631d-4fe4-8c97-18b1bd6edf8f None None] MFA Rules not processed for user `ff50e50587254bac912fb342a7e09891`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:39:14.964125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e3ec3ea8-631d-4fe4-8c97-18b1bd6edf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:39:15.037106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 752/1504] 10.222.0.49 () {58 vars in 1061 bytes} [Fri Sep 16 05:39:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:39:15.044772 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d988bb-680a-4a68-8df2-77ab04c34ecd None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:39:15.055098 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d988bb-680a-4a68-8df2-77ab04c34ecd None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:39:15.058542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21526b71-7e62-4bff-8ff0-7acbef12167b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:39:15.059119 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21526b71-7e62-4bff-8ff0-7acbef12167b service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:15.059391 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21526b71-7e62-4bff-8ff0-7acbef12167b service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:15.059652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21526b71-7e62-4bff-8ff0-7acbef12167b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:15.068088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21526b71-7e62-4bff-8ff0-7acbef12167b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:39:15.069573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21526b71-7e62-4bff-8ff0-7acbef12167b service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:39:15.078302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 753/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:39:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:39:15.122574 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6c3ef-2a72-49b7-97e9-93fd8f6ef5e3 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:15.122863 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6c3ef-2a72-49b7-97e9-93fd8f6ef5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:15.123135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6c3ef-2a72-49b7-97e9-93fd8f6ef5e3 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:15.124373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 753/1506] 10.222.0.49 () {52 vars in 830 bytes} [Fri Sep 16 05:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:39:15.127908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21526b71-7e62-4bff-8ff0-7acbef12167b service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:39:15.195201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21526b71-7e62-4bff-8ff0-7acbef12167b service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:39:15.199825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:39:15.200445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:15.200714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:15.200979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] PATH_INFO: `/v3/users/cb80e7671eb24d58977bfc432249aee4` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:15.202144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb80e7671eb24d58977bfc432249aee4)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:39:15.203209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: WARNING py.warnings [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:39:15.203209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: warnings.warn(msg) Sep 16 05:39:15.203209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: Sep 16 05:39:15.204428 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:39:15.396803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb80e7671eb24d58977bfc432249aee4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:39:15.404002 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb80e7671eb24d58977bfc432249aee4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:39:15.469508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb80e7671eb24d58977bfc432249aee4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:39:15.470155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb80e7671eb24d58977bfc432249aee4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:39:15.485496 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb80e7671eb24d58977bfc432249aee4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:39:15.507715 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:39:15.515723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 754/1507] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:39:15 2022] DELETE /identity/v3/users/cb80e7671eb24d58977bfc432249aee4 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:39:15.520132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145c943-af67-4666-8562-862f2a5d6945 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:15.520471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145c943-af67-4666-8562-862f2a5d6945 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:15.520849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145c943-af67-4666-8562-862f2a5d6945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:15.571276 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-3145c943-af67-4666-8562-862f2a5d6945 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:39:15.572546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3145c943-af67-4666-8562-862f2a5d6945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:39:15.581652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:39:15.631372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 754/1508] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:39:15.641021 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:39:15.642488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:39:15.720207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736b445-d51b-48ba-a41b-c763fc0e9b04 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:39:15.723707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260f71d-5c2f-4cea-ab92-0bdf75b7ab28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:39:15.724919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260f71d-5c2f-4cea-ab92-0bdf75b7ab28 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:15.725197 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260f71d-5c2f-4cea-ab92-0bdf75b7ab28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:15.725462 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260f71d-5c2f-4cea-ab92-0bdf75b7ab28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:15.734286 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260f71d-5c2f-4cea-ab92-0bdf75b7ab28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:39:15.735697 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3260f71d-5c2f-4cea-ab92-0bdf75b7ab28 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:39:15.745387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 755/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:39:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:39:16.226493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145c943-af67-4666-8562-862f2a5d6945 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:39:16.227956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3145c943-af67-4666-8562-862f2a5d6945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:39:16.240425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145c943-af67-4666-8562-862f2a5d6945 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:39:16.245144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182dd63d-879d-487d-9bc9-32c420e9c4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:39:16.246005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182dd63d-879d-487d-9bc9-32c420e9c4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:39:16.246166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182dd63d-879d-487d-9bc9-32c420e9c4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:39:16.246431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182dd63d-879d-487d-9bc9-32c420e9c4e9 admin admin] PATH_INFO: `/v3/projects/55d37540ac824e8d87afd8b8615ee1b4` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:39:16.247550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182dd63d-879d-487d-9bc9-32c420e9c4e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55d37540ac824e8d87afd8b8615ee1b4)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:39:16.248567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: WARNING py.warnings [None req-182dd63d-879d-487d-9bc9-32c420e9c4e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:39:16.248567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: warnings.warn(msg) Sep 16 05:39:16.248567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: Sep 16 05:39:16.249939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182dd63d-879d-487d-9bc9-32c420e9c4e9 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:39:16.375851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-182dd63d-879d-487d-9bc9-32c420e9c4e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55d37540ac824e8d87afd8b8615ee1b4', 'request_id': 'req-182dd63d-879d-487d-9bc9-32c420e9c4e9'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:39:16.382742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-182dd63d-879d-487d-9bc9-32c420e9c4e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55d37540ac824e8d87afd8b8615ee1b4', 'request_id': 'req-182dd63d-879d-487d-9bc9-32c420e9c4e9'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:39:16.397267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-182dd63d-879d-487d-9bc9-32c420e9c4e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55d37540ac824e8d87afd8b8615ee1b4', 'request_id': 'req-182dd63d-879d-487d-9bc9-32c420e9c4e9'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:39:16.475911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-182dd63d-879d-487d-9bc9-32c420e9c4e9 admin admin] The token cache is being invalidate because project 55d37540ac824e8d87afd8b8615ee1b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:39:16.476138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-182dd63d-879d-487d-9bc9-32c420e9c4e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:39:16.476763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-182dd63d-879d-487d-9bc9-32c420e9c4e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:39:16.478422 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 755/1510] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:39:16 2022] DELETE /identity/v3/projects/55d37540ac824e8d87afd8b8615ee1b4 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:39:25.741355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:39:26.471685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:40:38.935239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfdb46-3d16-4077-b949-82176ea44ad2 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:38.935738 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfdb46-3d16-4077-b949-82176ea44ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:38.935850 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfdb46-3d16-4077-b949-82176ea44ad2 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:38.937120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 05:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:40:38.942346 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8cd16-2952-4114-a7ff-9c13d7e8d8f4 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:38.942779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8cd16-2952-4114-a7ff-9c13d7e8d8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:38.942916 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8cd16-2952-4114-a7ff-9c13d7e8d8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:38.963663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-eaa8cd16-2952-4114-a7ff-9c13d7e8d8f4 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:40:38.964867 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-eaa8cd16-2952-4114-a7ff-9c13d7e8d8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:40:39.024013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 05:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:40:39.178907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfdb46-3d16-4077-b949-82176ea44ad2 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:40:39.180610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-efcfdb46-3d16-4077-b949-82176ea44ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:40:39.281951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfdb46-3d16-4077-b949-82176ea44ad2 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:40:39.287864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fe65a-7cfa-43e6-a077-34ba009faa15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:40:39.288461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fe65a-7cfa-43e6-a077-34ba009faa15 service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:39.288738 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fe65a-7cfa-43e6-a077-34ba009faa15 service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:39.289006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fe65a-7cfa-43e6-a077-34ba009faa15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:39.296908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fe65a-7cfa-43e6-a077-34ba009faa15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:40:39.298504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fe65a-7cfa-43e6-a077-34ba009faa15 service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:40:39.308793 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 757/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:40:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:40:40.394866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4ea8d-deea-4040-9610-ca785b4b75c5 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:40.395466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4ea8d-deea-4040-9610-ca785b4b75c5 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:40.395466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4ea8d-deea-4040-9610-ca785b4b75c5 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:40.396746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 05:40:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:40:40.399504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3084b-36c7-4075-9e7f-22e33add2178 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:40.399934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3084b-36c7-4075-9e7f-22e33add2178 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:40.400123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3084b-36c7-4075-9e7f-22e33add2178 None None] PATH_INFO: `/v3` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:40.401308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 758/1515] 10.222.0.15 () {54 vars in 925 bytes} [Fri Sep 16 05:40:40 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:40:40.403441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798ebe9-4d38-4a03-a5c5-dcd892499d3e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:40.403720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798ebe9-4d38-4a03-a5c5-dcd892499d3e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:40.403989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798ebe9-4d38-4a03-a5c5-dcd892499d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:40.407942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef235749-3da1-40c8-be63-a4567657ab8d None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:40.408223 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef235749-3da1-40c8-be63-a4567657ab8d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:40.408492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef235749-3da1-40c8-be63-a4567657ab8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:40.424010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-b798ebe9-4d38-4a03-a5c5-dcd892499d3e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:40:40.425474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b798ebe9-4d38-4a03-a5c5-dcd892499d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:40:40.433556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 758/1516] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 05:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:40:40.584060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-ef235749-3da1-40c8-be63-a4567657ab8d None None] MFA Rules not processed for user `9975c0dab76f4ccbb4d2107e8e780ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:40:40.585269 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ef235749-3da1-40c8-be63-a4567657ab8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:40:40.590166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798ebe9-4d38-4a03-a5c5-dcd892499d3e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:40:40.602278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798ebe9-4d38-4a03-a5c5-dcd892499d3e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:40:40.609470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5ccd9-0969-4ff0-bfac-accefa0f407a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:40:40.610268 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5ccd9-0969-4ff0-bfac-accefa0f407a service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:40.610722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5ccd9-0969-4ff0-bfac-accefa0f407a service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:40.610987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5ccd9-0969-4ff0-bfac-accefa0f407a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:40.619727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d5ccd9-0969-4ff0-bfac-accefa0f407a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:40:40.621311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d5ccd9-0969-4ff0-bfac-accefa0f407a service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:40:40.631627 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 759/1517] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:40:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:40:40.657872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 759/1518] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Sep 16 05:40:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:40:49.417663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f722bc8-25bf-494a-8002-5d47bc1ec662 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:49.418246 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f722bc8-25bf-494a-8002-5d47bc1ec662 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:49.418354 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f722bc8-25bf-494a-8002-5d47bc1ec662 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:49.419643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 760/1519] 10.222.0.49 () {54 vars in 961 bytes} [Fri Sep 16 05:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:40:49.424604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de856de-6cf2-4648-9cdd-75f1fd93bca3 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:49.425046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de856de-6cf2-4648-9cdd-75f1fd93bca3 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:49.425147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de856de-6cf2-4648-9cdd-75f1fd93bca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:49.444916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-0de856de-6cf2-4648-9cdd-75f1fd93bca3 None None] MFA Rules not processed for user `9975c0dab76f4ccbb4d2107e8e780ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:40:49.446322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0de856de-6cf2-4648-9cdd-75f1fd93bca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:40:49.454761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 760/1520] 10.222.0.49 () {58 vars in 1058 bytes} [Fri Sep 16 05:40:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:40:49.463222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f722bc8-25bf-494a-8002-5d47bc1ec662 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:40:49.476928 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f722bc8-25bf-494a-8002-5d47bc1ec662 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:40:49.481928 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85633f6-f454-4b45-bfc9-8636d24baee7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9975c0dab76f4ccbb4d2107e8e780ae5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:40:49.482567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85633f6-f454-4b45-bfc9-8636d24baee7 service placement] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:49.482843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85633f6-f454-4b45-bfc9-8636d24baee7 service placement] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:49.483109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85633f6-f454-4b45-bfc9-8636d24baee7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:49.491492 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85633f6-f454-4b45-bfc9-8636d24baee7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:40:49.493047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85633f6-f454-4b45-bfc9-8636d24baee7 service placement] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:40:49.502836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 761/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:40:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:40:52.126417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec23f4-43b2-419c-b27a-1da8a3d2a676 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:52.127001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec23f4-43b2-419c-b27a-1da8a3d2a676 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:52.127001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec23f4-43b2-419c-b27a-1da8a3d2a676 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:52.128311 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 05:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:40:52.135141 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c24b0b-76d6-4185-9fa5-f19f165cb8d9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:52.135603 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c24b0b-76d6-4185-9fa5-f19f165cb8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:52.135713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c24b0b-76d6-4185-9fa5-f19f165cb8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:52.155530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-a7c24b0b-76d6-4185-9fa5-f19f165cb8d9 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:40:52.156700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a7c24b0b-76d6-4185-9fa5-f19f165cb8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:40:52.165092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 05:40:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:40:52.326291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec23f4-43b2-419c-b27a-1da8a3d2a676 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:40:52.341180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec23f4-43b2-419c-b27a-1da8a3d2a676 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:40:52.346674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021dc34-676d-4c74-bce7-cfe040d2e090 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:40:52.347261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021dc34-676d-4c74-bce7-cfe040d2e090 service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:52.347534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021dc34-676d-4c74-bce7-cfe040d2e090 service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:52.347800 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021dc34-676d-4c74-bce7-cfe040d2e090 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:52.356655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021dc34-676d-4c74-bce7-cfe040d2e090 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:40:52.363229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021dc34-676d-4c74-bce7-cfe040d2e090 service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:40:52.369386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 762/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:40:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:40:53.458057 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc2e5f-0c5e-47f4-a28a-9795de813a8d None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:53.458589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc2e5f-0c5e-47f4-a28a-9795de813a8d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:53.458589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc2e5f-0c5e-47f4-a28a-9795de813a8d None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:53.460101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 16 05:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:40:53.466396 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25270d66-e00e-4ae7-b384-e583c130822f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:53.466837 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25270d66-e00e-4ae7-b384-e583c130822f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:53.466947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25270d66-e00e-4ae7-b384-e583c130822f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:53.474407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:40:53.486901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-25270d66-e00e-4ae7-b384-e583c130822f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:40:53.488075 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-25270d66-e00e-4ae7-b384-e583c130822f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:40:53.496441 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 16 05:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:40:53.650188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc2e5f-0c5e-47f4-a28a-9795de813a8d None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:40:53.665207 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc2e5f-0c5e-47f4-a28a-9795de813a8d None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:40:53.670806 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2eb381-cf52-45b1-b78f-13834cf71ebc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:40:53.671397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2eb381-cf52-45b1-b78f-13834cf71ebc service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:53.671668 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2eb381-cf52-45b1-b78f-13834cf71ebc service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:53.671933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2eb381-cf52-45b1-b78f-13834cf71ebc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:53.680861 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2eb381-cf52-45b1-b78f-13834cf71ebc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:40:53.682452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2eb381-cf52-45b1-b78f-13834cf71ebc service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:40:53.691751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 764/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:40:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:40:57.606911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325af7fd-9465-4a19-8e4d-4b0a6fc3b2f2 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:57.607450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325af7fd-9465-4a19-8e4d-4b0a6fc3b2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:57.607450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325af7fd-9465-4a19-8e4d-4b0a6fc3b2f2 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:57.608680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:40:57.613755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406f6d-6e36-4ceb-8aec-71ce109bf12c None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:57.614204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406f6d-6e36-4ceb-8aec-71ce109bf12c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:57.614348 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d406f6d-6e36-4ceb-8aec-71ce109bf12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:57.635391 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-2d406f6d-6e36-4ceb-8aec-71ce109bf12c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:40:57.636587 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2d406f6d-6e36-4ceb-8aec-71ce109bf12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:40:57.644362 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:40:57.936417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325af7fd-9465-4a19-8e4d-4b0a6fc3b2f2 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:40:57.938095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-325af7fd-9465-4a19-8e4d-4b0a6fc3b2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:40:57.993348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325af7fd-9465-4a19-8e4d-4b0a6fc3b2f2 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:40:57.996809 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3a2b2-e6fe-4958-8f93-90b5c618f692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:40:57.997398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3a2b2-e6fe-4958-8f93-90b5c618f692 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:40:57.997668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3a2b2-e6fe-4958-8f93-90b5c618f692 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:40:57.997959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3a2b2-e6fe-4958-8f93-90b5c618f692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:40:58.007031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3a2b2-e6fe-4958-8f93-90b5c618f692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:40:58.008428 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3a2b2-e6fe-4958-8f93-90b5c618f692 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:40:58.017437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 765/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:41:00.199258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cadbfea-54bd-473b-b5b7-8fc7c72140a3 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:41:00.199808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cadbfea-54bd-473b-b5b7-8fc7c72140a3 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:41:00.199808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cadbfea-54bd-473b-b5b7-8fc7c72140a3 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:41:00.201047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:41:00.206404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2700b-3be1-4947-b9e6-2d8c0be7eb99 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:41:00.206860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2700b-3be1-4947-b9e6-2d8c0be7eb99 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:41:00.206966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b2700b-3be1-4947-b9e6-2d8c0be7eb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:41:00.227767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-b5b2700b-3be1-4947-b9e6-2d8c0be7eb99 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:41:00.228947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b5b2700b-3be1-4947-b9e6-2d8c0be7eb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:41:00.236732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:41:00.527152 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cadbfea-54bd-473b-b5b7-8fc7c72140a3 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:41:00.539000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cadbfea-54bd-473b-b5b7-8fc7c72140a3 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:41:00.542477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de00bc5-8916-40bf-9b4c-54f530ae0f15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:41:00.543081 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de00bc5-8916-40bf-9b4c-54f530ae0f15 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:41:00.543350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de00bc5-8916-40bf-9b4c-54f530ae0f15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:41:00.543614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de00bc5-8916-40bf-9b4c-54f530ae0f15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:41:00.553286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de00bc5-8916-40bf-9b4c-54f530ae0f15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:41:00.554735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de00bc5-8916-40bf-9b4c-54f530ae0f15 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:41:00.564452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 767/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:41:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:41:04.264925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1667b4a-2b4f-4982-b42f-301c5a63a5a5 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:41:04.265448 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1667b4a-2b4f-4982-b42f-301c5a63a5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:41:04.265448 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1667b4a-2b4f-4982-b42f-301c5a63a5a5 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:41:04.266934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:41:04.272913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba76d-7398-443b-8f67-e3be04269b85 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:41:04.273321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba76d-7398-443b-8f67-e3be04269b85 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:41:04.273483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba76d-7398-443b-8f67-e3be04269b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:41:04.293498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f0fba76d-7398-443b-8f67-e3be04269b85 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:41:04.294704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f0fba76d-7398-443b-8f67-e3be04269b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:41:04.302780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:41:04.619129 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1667b4a-2b4f-4982-b42f-301c5a63a5a5 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:41:04.620909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a1667b4a-2b4f-4982-b42f-301c5a63a5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:41:04.677317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1667b4a-2b4f-4982-b42f-301c5a63a5a5 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:41:04.682922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5447e-8b54-427d-95fc-7f9b072ffcb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:41:04.683941 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5447e-8b54-427d-95fc-7f9b072ffcb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:41:04.684420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5447e-8b54-427d-95fc-7f9b072ffcb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:41:04.684873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5447e-8b54-427d-95fc-7f9b072ffcb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:41:04.698850 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5447e-8b54-427d-95fc-7f9b072ffcb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:41:04.700292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5447e-8b54-427d-95fc-7f9b072ffcb2 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:41:04.714348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 768/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:41:06.904288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2e4f0-e55a-422d-bbfa-9419f0f13b84 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:41:06.904835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2e4f0-e55a-422d-bbfa-9419f0f13b84 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:41:06.904835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2e4f0-e55a-422d-bbfa-9419f0f13b84 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:41:06.906214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 16 05:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:41:06.911627 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1333f-1fce-4db7-a8b3-cbb50f061556 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:41:06.912617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1333f-1fce-4db7-a8b3-cbb50f061556 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:41:06.912816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1333f-1fce-4db7-a8b3-cbb50f061556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:41:06.932081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-e9c1333f-1fce-4db7-a8b3-cbb50f061556 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:41:06.933257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e9c1333f-1fce-4db7-a8b3-cbb50f061556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:41:06.940917 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 16 05:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:41:07.249732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2e4f0-e55a-422d-bbfa-9419f0f13b84 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:41:07.251427 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-46b2e4f0-e55a-422d-bbfa-9419f0f13b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:41:07.259682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=64 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:41:07.265390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2e4f0-e55a-422d-bbfa-9419f0f13b84 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:41:07.268861 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30570fd2-2c3a-41f6-b204-21230e95dd30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:41:07.269437 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30570fd2-2c3a-41f6-b204-21230e95dd30 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:41:07.269711 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30570fd2-2c3a-41f6-b204-21230e95dd30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:41:07.270011 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30570fd2-2c3a-41f6-b204-21230e95dd30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:41:07.279727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30570fd2-2c3a-41f6-b204-21230e95dd30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:41:07.285092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30570fd2-2c3a-41f6-b204-21230e95dd30 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:41:07.290833 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 770/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:41:16.927004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=60 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:41:17.303773 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=5 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:29.525322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18e4e2-b85f-403c-b680-6268bafd14c8 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:29.526381 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18e4e2-b85f-403c-b680-6268bafd14c8 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:29.526819 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18e4e2-b85f-403c-b680-6268bafd14c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:29.559065 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-0e18e4e2-b85f-403c-b680-6268bafd14c8 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:29.560295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0e18e4e2-b85f-403c-b680-6268bafd14c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:29.580398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 770/1540] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:29.589485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2066a86-0d24-45bb-997e-cf2e9968a5d2 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:29.590561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2066a86-0d24-45bb-997e-cf2e9968a5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:29.591062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2066a86-0d24-45bb-997e-cf2e9968a5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:29.622225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea0426-4195-4bcc-9617-69f68f048397 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:29.622549 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea0426-4195-4bcc-9617-69f68f048397 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:29.622767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea0426-4195-4bcc-9617-69f68f048397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:29.627890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-d2066a86-0d24-45bb-997e-cf2e9968a5d2 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:29.629290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d2066a86-0d24-45bb-997e-cf2e9968a5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:29.643732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 771/1541] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:29.645580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-f9ea0426-4195-4bcc-9617-69f68f048397 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:29.650970 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f9ea0426-4195-4bcc-9617-69f68f048397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:29.655024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2066a86-0d24-45bb-997e-cf2e9968a5d2 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:29.661714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 771/1542] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:29.665774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2066a86-0d24-45bb-997e-cf2e9968a5d2 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:29.678780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cc101-8bc7-4f51-b883-7260433f80b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:29.679462 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cc101-8bc7-4f51-b883-7260433f80b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:29.679748 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cc101-8bc7-4f51-b883-7260433f80b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:29.679999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cc101-8bc7-4f51-b883-7260433f80b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:29.680849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cc101-8bc7-4f51-b883-7260433f80b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:42:29.681149 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cc101-8bc7-4f51-b883-7260433f80b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:29.682217 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca74ca-b6b0-47ea-baeb-4068169f1438 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:29.682550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca74ca-b6b0-47ea-baeb-4068169f1438 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:29.682828 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca74ca-b6b0-47ea-baeb-4068169f1438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:29.683009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cc101-8bc7-4f51-b883-7260433f80b4 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:29.702254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 772/1543] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:42:29.718402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cc101-8bc7-4f51-b883-7260433f80b4 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:29.724385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-39ca74ca-b6b0-47ea-baeb-4068169f1438 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:29.725612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-39ca74ca-b6b0-47ea-baeb-4068169f1438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:29.734172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cc101-8bc7-4f51-b883-7260433f80b4 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:29.741277 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71c63a-afe1-4cb1-aed3-aa833de2f5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:29.741939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71c63a-afe1-4cb1-aed3-aa833de2f5ee admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:29.742275 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71c63a-afe1-4cb1-aed3-aa833de2f5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:29.742545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71c63a-afe1-4cb1-aed3-aa833de2f5ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:29.743244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71c63a-afe1-4cb1-aed3-aa833de2f5ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:29.744421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: WARNING py.warnings [None req-4a71c63a-afe1-4cb1-aed3-aa833de2f5ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:42:29.744421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: warnings.warn(msg) Sep 16 05:42:29.744421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: Sep 16 05:42:29.744717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 772/1544] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:42:29.745661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71c63a-afe1-4cb1-aed3-aa833de2f5ee admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:29.748931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca74ca-b6b0-47ea-baeb-4068169f1438 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:29.767389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca74ca-b6b0-47ea-baeb-4068169f1438 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:29.777445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fe11d-cfc1-458c-9001-d0265283021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:29.778371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fe11d-cfc1-458c-9001-d0265283021a admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:29.778645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fe11d-cfc1-458c-9001-d0265283021a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:29.778918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fe11d-cfc1-458c-9001-d0265283021a admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:29.779576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fe11d-cfc1-458c-9001-d0265283021a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:42:29.779880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fe11d-cfc1-458c-9001-d0265283021a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:29.780913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: WARNING py.warnings [None req-ad2fe11d-cfc1-458c-9001-d0265283021a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:42:29.780913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: warnings.warn(msg) Sep 16 05:42:29.780913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: Sep 16 05:42:29.781805 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fe11d-cfc1-458c-9001-d0265283021a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:29.805044 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 773/1545] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:42:29.813249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fe11d-cfc1-458c-9001-d0265283021a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:29.827293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fe11d-cfc1-458c-9001-d0265283021a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:29.841282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999737d-4fe5-4265-b1a4-5e31ac50ad19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:29.841938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999737d-4fe5-4265-b1a4-5e31ac50ad19 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:29.842220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999737d-4fe5-4265-b1a4-5e31ac50ad19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:29.842488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999737d-4fe5-4265-b1a4-5e31ac50ad19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:29.843231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999737d-4fe5-4265-b1a4-5e31ac50ad19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:29.845577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999737d-4fe5-4265-b1a4-5e31ac50ad19 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:29.890405 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 773/1546] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:29 2022] POST /identity/v3/projects => generated 357 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:42:29.891722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a5662-63da-4360-8da1-f784689b9741 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:29.892007 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a5662-63da-4360-8da1-f784689b9741 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:29.892278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a5662-63da-4360-8da1-f784689b9741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:29.926035 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-a16a5662-63da-4360-8da1-f784689b9741 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:29.927423 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a16a5662-63da-4360-8da1-f784689b9741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:29.955768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 774/1547] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:29 2022] POST /identity/v3/projects => generated 393 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:29.957176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37960e16-e31e-4fda-90d6-3447ffccb016 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:29.957575 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37960e16-e31e-4fda-90d6-3447ffccb016 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:29.957861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37960e16-e31e-4fda-90d6-3447ffccb016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:29.995209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-37960e16-e31e-4fda-90d6-3447ffccb016 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:29.996429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-37960e16-e31e-4fda-90d6-3447ffccb016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:30.024274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 774/1548] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:30.025114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a5662-63da-4360-8da1-f784689b9741 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.035446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a5662-63da-4360-8da1-f784689b9741 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.071271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 775/1549] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:30.072526 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895ee2b-40a7-42c1-9895-68f794eb1ce7 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.072807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895ee2b-40a7-42c1-9895-68f794eb1ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.073077 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895ee2b-40a7-42c1-9895-68f794eb1ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.091901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-5895ee2b-40a7-42c1-9895-68f794eb1ce7 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:30.093098 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5895ee2b-40a7-42c1-9895-68f794eb1ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:30.101615 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 776/1550] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:30.102396 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895ee2b-40a7-42c1-9895-68f794eb1ce7 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.104274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b55410-ca7e-4478-9257-985ab01b30f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.104952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b55410-ca7e-4478-9257-985ab01b30f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.105290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b55410-ca7e-4478-9257-985ab01b30f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.105617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b55410-ca7e-4478-9257-985ab01b30f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.106329 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b55410-ca7e-4478-9257-985ab01b30f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.108291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b55410-ca7e-4478-9257-985ab01b30f8 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.112140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895ee2b-40a7-42c1-9895-68f794eb1ce7 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.117382 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427cd8f7-0cba-4702-b465-082b2bff434d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.118034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cd8f7-0cba-4702-b465-082b2bff434d admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.118307 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cd8f7-0cba-4702-b465-082b2bff434d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.118569 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427cd8f7-0cba-4702-b465-082b2bff434d admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.119196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427cd8f7-0cba-4702-b465-082b2bff434d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.120185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: WARNING py.warnings [None req-427cd8f7-0cba-4702-b465-082b2bff434d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:42:30.120185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: warnings.warn(msg) Sep 16 05:42:30.120185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: Sep 16 05:42:30.121364 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427cd8f7-0cba-4702-b465-082b2bff434d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.209938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 775/1551] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/users => generated 279 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:30.211118 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cd3a4-abac-4601-ad37-a6b437426bc9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.211403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cd3a4-abac-4601-ad37-a6b437426bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.211722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90cd3a4-abac-4601-ad37-a6b437426bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.231060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 777/1552] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/users => generated 297 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:30.231060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e91a0-2d6e-4bfe-b7f3-1d2f7c4d29f0 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.231060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e91a0-2d6e-4bfe-b7f3-1d2f7c4d29f0 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.231060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e91a0-2d6e-4bfe-b7f3-1d2f7c4d29f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.234321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-e90cd3a4-abac-4601-ad37-a6b437426bc9 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:30.235544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e90cd3a4-abac-4601-ad37-a6b437426bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:30.243591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 776/1553] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:30.248017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a8abb-2418-402b-a7bb-6df02861f362 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.248017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a8abb-2418-402b-a7bb-6df02861f362 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.248017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a8abb-2418-402b-a7bb-6df02861f362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.254649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-020e91a0-2d6e-4bfe-b7f3-1d2f7c4d29f0 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:30.256001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-020e91a0-2d6e-4bfe-b7f3-1d2f7c4d29f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:30.264105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 778/1554] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:30.265304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-761a8abb-2418-402b-a7bb-6df02861f362 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:30.265751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73830fe-9ae7-4ff3-90ed-9680d29ff164 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.266235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73830fe-9ae7-4ff3-90ed-9680d29ff164 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.266546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-761a8abb-2418-402b-a7bb-6df02861f362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:30.266857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73830fe-9ae7-4ff3-90ed-9680d29ff164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.275658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 777/1555] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:30.277061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eddf25-8329-4208-b46e-fe4971c8cd77 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.277431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eddf25-8329-4208-b46e-fe4971c8cd77 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.278210 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eddf25-8329-4208-b46e-fe4971c8cd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.288294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-e73830fe-9ae7-4ff3-90ed-9680d29ff164 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:30.289464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e73830fe-9ae7-4ff3-90ed-9680d29ff164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:30.297112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 779/1556] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:30.298209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c8eddf25-8329-4208-b46e-fe4971c8cd77 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:30.298340 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1a06f-ffb9-438c-a3ab-ddb00e499dfe None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.298573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1a06f-ffb9-438c-a3ab-ddb00e499dfe None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.298835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1a06f-ffb9-438c-a3ab-ddb00e499dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.299409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c8eddf25-8329-4208-b46e-fe4971c8cd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:30.308315 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 778/1557] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:30.309112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eddf25-8329-4208-b46e-fe4971c8cd77 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.318242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-35e1a06f-ffb9-438c-a3ab-ddb00e499dfe None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:30.318855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eddf25-8329-4208-b46e-fe4971c8cd77 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.319418 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-35e1a06f-ffb9-438c-a3ab-ddb00e499dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:30.323331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637d746-2dfe-4caf-8c0d-3954102a4cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.323910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637d746-2dfe-4caf-8c0d-3954102a4cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.324178 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637d746-2dfe-4caf-8c0d-3954102a4cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.324436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637d746-2dfe-4caf-8c0d-3954102a4cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.325058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637d746-2dfe-4caf-8c0d-3954102a4cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.326655 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637d746-2dfe-4caf-8c0d-3954102a4cb9 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.327083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 780/1558] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:30.327896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1a06f-ffb9-438c-a3ab-ddb00e499dfe None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.337741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1a06f-ffb9-438c-a3ab-ddb00e499dfe None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.337914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 779/1559] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:30.341754 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789acfe0-05fe-4b8f-8f24-70d5021268d7 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.342063 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789acfe0-05fe-4b8f-8f24-70d5021268d7 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.342330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789acfe0-05fe-4b8f-8f24-70d5021268d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.342449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6a2c3-ea14-4399-b54d-aa475c94d7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.343011 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6a2c3-ea14-4399-b54d-aa475c94d7b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.343274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6a2c3-ea14-4399-b54d-aa475c94d7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.343663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6a2c3-ea14-4399-b54d-aa475c94d7b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.344334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6a2c3-ea14-4399-b54d-aa475c94d7b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.346054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6a2c3-ea14-4399-b54d-aa475c94d7b0 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.356980 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 781/1560] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:30.358303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e602d-196b-41df-8f35-5c7c342ea1fa None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.358585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e602d-196b-41df-8f35-5c7c342ea1fa None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.358852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e602d-196b-41df-8f35-5c7c342ea1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.362763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-789acfe0-05fe-4b8f-8f24-70d5021268d7 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:30.364028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-789acfe0-05fe-4b8f-8f24-70d5021268d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:30.372864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 780/1561] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:30.377134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789acfe0-05fe-4b8f-8f24-70d5021268d7 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.377883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-949e602d-196b-41df-8f35-5c7c342ea1fa None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:30.379049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-949e602d-196b-41df-8f35-5c7c342ea1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:30.386502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789acfe0-05fe-4b8f-8f24-70d5021268d7 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.387097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 782/1562] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:30.388201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87250520-2427-4bcd-93ab-8599e5e97477 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.388473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87250520-2427-4bcd-93ab-8599e5e97477 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.388736 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87250520-2427-4bcd-93ab-8599e5e97477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.390976 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b2bb9-82f9-462c-8181-41c1e75ecac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.391556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b2bb9-82f9-462c-8181-41c1e75ecac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.391827 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b2bb9-82f9-462c-8181-41c1e75ecac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.392086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b2bb9-82f9-462c-8181-41c1e75ecac0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.392712 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b2bb9-82f9-462c-8181-41c1e75ecac0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:42:30.393002 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b2bb9-82f9-462c-8181-41c1e75ecac0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.394622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81b2bb9-82f9-462c-8181-41c1e75ecac0 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.402157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:30.408801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-87250520-2427-4bcd-93ab-8599e5e97477 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:30.409000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 781/1563] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:42:30.409926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b2bb9-82f9-462c-8181-41c1e75ecac0 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.410675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-87250520-2427-4bcd-93ab-8599e5e97477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:30.420303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 783/1564] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:30.420812 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81b2bb9-82f9-462c-8181-41c1e75ecac0 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.422614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87250520-2427-4bcd-93ab-8599e5e97477 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.425475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a6e63-3287-41bb-9f77-a6ef4dabd7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.426094 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a6e63-3287-41bb-9f77-a6ef4dabd7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.426940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a6e63-3287-41bb-9f77-a6ef4dabd7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.427204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a6e63-3287-41bb-9f77-a6ef4dabd7f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.427837 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a6e63-3287-41bb-9f77-a6ef4dabd7f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:42:30.428127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a6e63-3287-41bb-9f77-a6ef4dabd7f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.429733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2a6e63-3287-41bb-9f77-a6ef4dabd7f0 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.432425 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87250520-2427-4bcd-93ab-8599e5e97477 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.436924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b8997-c520-4a0a-ae60-0d188c001a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.437510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b8997-c520-4a0a-ae60-0d188c001a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.437784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b8997-c520-4a0a-ae60-0d188c001a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.438153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b8997-c520-4a0a-ae60-0d188c001a26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.438840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b8997-c520-4a0a-ae60-0d188c001a26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:42:30.439128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b8997-c520-4a0a-ae60-0d188c001a26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.440687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b8997-c520-4a0a-ae60-0d188c001a26 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.442985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 782/1565] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:42:30.443873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a6e63-3287-41bb-9f77-a6ef4dabd7f0 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.458102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 784/1566] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:42:30.458274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2a6e63-3287-41bb-9f77-a6ef4dabd7f0 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.458385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b8997-c520-4a0a-ae60-0d188c001a26 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.461285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6c7a3-44b2-4d57-aee3-7bff693ccbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.462077 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6c7a3-44b2-4d57-aee3-7bff693ccbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.462568 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6c7a3-44b2-4d57-aee3-7bff693ccbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.462932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6c7a3-44b2-4d57-aee3-7bff693ccbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.463842 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6c7a3-44b2-4d57-aee3-7bff693ccbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.465424 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a6c7a3-44b2-4d57-aee3-7bff693ccbf2 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.467343 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b8997-c520-4a0a-ae60-0d188c001a26 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.471804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12958d-91fb-4046-8543-565d24f20cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.472391 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12958d-91fb-4046-8543-565d24f20cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.472671 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12958d-91fb-4046-8543-565d24f20cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.472932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12958d-91fb-4046-8543-565d24f20cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.473879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12958d-91fb-4046-8543-565d24f20cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.475432 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12958d-91fb-4046-8543-565d24f20cc6 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.479249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 783/1567] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:30.479249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6c7a3-44b2-4d57-aee3-7bff693ccbf2 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.485838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 785/1568] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:30.486684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12958d-91fb-4046-8543-565d24f20cc6 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.487147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6c7a3-44b2-4d57-aee3-7bff693ccbf2 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.492557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16b0f9-f8fe-4cbe-b256-4eed7b2603dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.493150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16b0f9-f8fe-4cbe-b256-4eed7b2603dc admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.493423 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16b0f9-f8fe-4cbe-b256-4eed7b2603dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.493692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb16b0f9-f8fe-4cbe-b256-4eed7b2603dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.494371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16b0f9-f8fe-4cbe-b256-4eed7b2603dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:42:30.495031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16b0f9-f8fe-4cbe-b256-4eed7b2603dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.496629 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb16b0f9-f8fe-4cbe-b256-4eed7b2603dc admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.496767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12958d-91fb-4046-8543-565d24f20cc6 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.501164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8108584-171c-4c0c-a92b-98d5d692a065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.501749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8108584-171c-4c0c-a92b-98d5d692a065 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.502056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8108584-171c-4c0c-a92b-98d5d692a065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.502493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8108584-171c-4c0c-a92b-98d5d692a065 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.504143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8108584-171c-4c0c-a92b-98d5d692a065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:42:30.504143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8108584-171c-4c0c-a92b-98d5d692a065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.506092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8108584-171c-4c0c-a92b-98d5d692a065 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.510158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 784/1569] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:42:30.511022 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16b0f9-f8fe-4cbe-b256-4eed7b2603dc admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.515150 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:30.518626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 786/1570] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:42:30.519378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8108584-171c-4c0c-a92b-98d5d692a065 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.523416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb16b0f9-f8fe-4cbe-b256-4eed7b2603dc admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.528426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d9fb8-e5e6-4cb7-abd2-48a146fe4ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.529021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d9fb8-e5e6-4cb7-abd2-48a146fe4ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.529298 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d9fb8-e5e6-4cb7-abd2-48a146fe4ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.529566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d9fb8-e5e6-4cb7-abd2-48a146fe4ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.530229 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d9fb8-e5e6-4cb7-abd2-48a146fe4ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.531495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8108584-171c-4c0c-a92b-98d5d692a065 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.532158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d9fb8-e5e6-4cb7-abd2-48a146fe4ae2 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.536120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e2593-6d07-42fd-bc13-4b0bd9df3868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.536803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e2593-6d07-42fd-bc13-4b0bd9df3868 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.537076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e2593-6d07-42fd-bc13-4b0bd9df3868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.537338 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e2593-6d07-42fd-bc13-4b0bd9df3868 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.538002 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e2593-6d07-42fd-bc13-4b0bd9df3868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:42:30.538294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e2593-6d07-42fd-bc13-4b0bd9df3868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.539886 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e2593-6d07-42fd-bc13-4b0bd9df3868 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.553040 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 787/1571] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:42:30.553988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e2593-6d07-42fd-bc13-4b0bd9df3868 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.563799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e2593-6d07-42fd-bc13-4b0bd9df3868 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.568856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dede9-e5bf-456e-b764-86702d7e793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.569459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dede9-e5bf-456e-b764-86702d7e793a admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.569731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dede9-e5bf-456e-b764-86702d7e793a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.570026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dede9-e5bf-456e-b764-86702d7e793a admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.570645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3dede9-e5bf-456e-b764-86702d7e793a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.572507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3dede9-e5bf-456e-b764-86702d7e793a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.628550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 785/1572] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/projects => generated 389 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:30.629704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d9fb8-e5e6-4cb7-abd2-48a146fe4ae2 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.654941 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 788/1573] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:30.654941 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dede9-e5bf-456e-b764-86702d7e793a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.694147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d9fb8-e5e6-4cb7-abd2-48a146fe4ae2 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.715494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dede9-e5bf-456e-b764-86702d7e793a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.720058 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63dbb4-eed0-4a74-be0b-2c732f22a7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.720717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63dbb4-eed0-4a74-be0b-2c732f22a7b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.720995 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63dbb4-eed0-4a74-be0b-2c732f22a7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.721269 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63dbb4-eed0-4a74-be0b-2c732f22a7b2 admin admin] PATH_INFO: `/v3/projects/c25ae3f893dd4d2eb1da0641addded20/users/89299b0f46da4796954983b30a37cf2e/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.746414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63dbb4-eed0-4a74-be0b-2c732f22a7b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c25ae3f893dd4d2eb1da0641addded20, user_id=89299b0f46da4796954983b30a37cf2e, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.749373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63dbb4-eed0-4a74-be0b-2c732f22a7b2 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.753872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e8a9c-380b-4280-9e79-874b37ddbaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.754695 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e8a9c-380b-4280-9e79-874b37ddbaee admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.754695 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e8a9c-380b-4280-9e79-874b37ddbaee admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.755409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e8a9c-380b-4280-9e79-874b37ddbaee admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.755874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e8a9c-380b-4280-9e79-874b37ddbaee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.757773 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0e8a9c-380b-4280-9e79-874b37ddbaee admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.810664 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 789/1574] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:30 2022] PUT /identity/v3/projects/c25ae3f893dd4d2eb1da0641addded20/users/89299b0f46da4796954983b30a37cf2e/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:30.811468 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63dbb4-eed0-4a74-be0b-2c732f22a7b2 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.869004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 786/1575] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:30.869949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e8a9c-380b-4280-9e79-874b37ddbaee admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:30.872939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63dbb4-eed0-4a74-be0b-2c732f22a7b2 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.906060 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:30.924708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:30.932201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b409953-5d47-4f3d-ad4b-473b1eb59571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.932868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b409953-5d47-4f3d-ad4b-473b1eb59571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.933145 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b409953-5d47-4f3d-ad4b-473b1eb59571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.933413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b409953-5d47-4f3d-ad4b-473b1eb59571 admin admin] PATH_INFO: `/v3/projects/d6a0481b87f34225a6a44f702ba75f68/users/12e7e80742cc426da9984af75016a890/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.940186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e8a9c-380b-4280-9e79-874b37ddbaee admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:30.944993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b7d3e-e9ee-4027-8d3f-99c41ac96686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:30.945711 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b7d3e-e9ee-4027-8d3f-99c41ac96686 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:30.946512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b7d3e-e9ee-4027-8d3f-99c41ac96686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:30.947044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b7d3e-e9ee-4027-8d3f-99c41ac96686 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:30.947864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b409953-5d47-4f3d-ad4b-473b1eb59571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6a0481b87f34225a6a44f702ba75f68, user_id=12e7e80742cc426da9984af75016a890, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.948190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b7d3e-e9ee-4027-8d3f-99c41ac96686 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:30.950135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b7d3e-e9ee-4027-8d3f-99c41ac96686 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:30.950539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b409953-5d47-4f3d-ad4b-473b1eb59571 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.033391 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 790/1576] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:30 2022] PUT /identity/v3/projects/d6a0481b87f34225a6a44f702ba75f68/users/12e7e80742cc426da9984af75016a890/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:31.034296 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b409953-5d47-4f3d-ad4b-473b1eb59571 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.064978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 787/1577] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:30 2022] POST /identity/v3/projects => generated 403 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:31.074037 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b7d3e-e9ee-4027-8d3f-99c41ac96686 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.097203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b409953-5d47-4f3d-ad4b-473b1eb59571 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.127469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b7d3e-e9ee-4027-8d3f-99c41ac96686 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.132303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d3183-edef-4f19-a86f-3effabb68d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.132923 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d3183-edef-4f19-a86f-3effabb68d26 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.133192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d3183-edef-4f19-a86f-3effabb68d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.133457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d3183-edef-4f19-a86f-3effabb68d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.134106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d3183-edef-4f19-a86f-3effabb68d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.136015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d3183-edef-4f19-a86f-3effabb68d26 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.153684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359ca06-e691-4094-ac69-1952c66b4802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.154331 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359ca06-e691-4094-ac69-1952c66b4802 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.154601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359ca06-e691-4094-ac69-1952c66b4802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.154864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359ca06-e691-4094-ac69-1952c66b4802 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.155481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359ca06-e691-4094-ac69-1952c66b4802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.157360 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359ca06-e691-4094-ac69-1952c66b4802 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.262982 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 788/1578] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:31 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:31.263994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d3183-edef-4f19-a86f-3effabb68d26 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.275630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 791/1579] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:31 2022] POST /identity/v3/projects => generated 377 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:31.277003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359ca06-e691-4094-ac69-1952c66b4802 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.328789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d3183-edef-4f19-a86f-3effabb68d26 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.340068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359ca06-e691-4094-ac69-1952c66b4802 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.344740 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be4113-eaf1-46b5-89d3-817b5fa32f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.345332 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be4113-eaf1-46b5-89d3-817b5fa32f0b admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.345600 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be4113-eaf1-46b5-89d3-817b5fa32f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.345889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be4113-eaf1-46b5-89d3-817b5fa32f0b admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.346507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78be4113-eaf1-46b5-89d3-817b5fa32f0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.348428 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78be4113-eaf1-46b5-89d3-817b5fa32f0b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.374221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:31.386613 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c8aa4-67d0-47e1-8149-7de0458a1f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.387226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c8aa4-67d0-47e1-8149-7de0458a1f99 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.387491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c8aa4-67d0-47e1-8149-7de0458a1f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.387757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c8aa4-67d0-47e1-8149-7de0458a1f99 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.388375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c8aa4-67d0-47e1-8149-7de0458a1f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.390305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c8aa4-67d0-47e1-8149-7de0458a1f99 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.434383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:31.439379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 792/1580] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:31 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:31.440675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34105d1d-f128-41e3-b787-006d44ef4332 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.440964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34105d1d-f128-41e3-b787-006d44ef4332 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.441223 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34105d1d-f128-41e3-b787-006d44ef4332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.460384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 789/1581] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:31 2022] POST /identity/v3/users => generated 295 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:31.460384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c8aa4-67d0-47e1-8149-7de0458a1f99 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.469604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c8aa4-67d0-47e1-8149-7de0458a1f99 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.470726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-34105d1d-f128-41e3-b787-006d44ef4332 None None] MFA Rules not processed for user `89299b0f46da4796954983b30a37cf2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:31.471934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-34105d1d-f128-41e3-b787-006d44ef4332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:31.474387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491a6eb-8a2f-429d-b8bd-a64e321875eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.474977 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491a6eb-8a2f-429d-b8bd-a64e321875eb admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.475244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491a6eb-8a2f-429d-b8bd-a64e321875eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.475506 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491a6eb-8a2f-429d-b8bd-a64e321875eb admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.476120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491a6eb-8a2f-429d-b8bd-a64e321875eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.478109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491a6eb-8a2f-429d-b8bd-a64e321875eb admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.538298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 793/1582] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:31.542507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19264cd6-7cc4-42dd-8579-43c3e4bf2f83 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.542507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19264cd6-7cc4-42dd-8579-43c3e4bf2f83 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.542507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19264cd6-7cc4-42dd-8579-43c3e4bf2f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.546215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 790/1583] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:31 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:31.547046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491a6eb-8a2f-429d-b8bd-a64e321875eb admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.557873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491a6eb-8a2f-429d-b8bd-a64e321875eb admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.559563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-19264cd6-7cc4-42dd-8579-43c3e4bf2f83 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:31.560776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-19264cd6-7cc4-42dd-8579-43c3e4bf2f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:31.562835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828940f-9125-4f8e-95a8-206adef50f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.563399 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828940f-9125-4f8e-95a8-206adef50f8f admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.563903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828940f-9125-4f8e-95a8-206adef50f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.564366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828940f-9125-4f8e-95a8-206adef50f8f admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.565079 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828940f-9125-4f8e-95a8-206adef50f8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.568379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828940f-9125-4f8e-95a8-206adef50f8f admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.571662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 794/1584] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:31.574551 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19264cd6-7cc4-42dd-8579-43c3e4bf2f83 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.585546 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19264cd6-7cc4-42dd-8579-43c3e4bf2f83 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.590857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bea84e-3934-478e-92b7-54ad29c6696a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.596154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bea84e-3934-478e-92b7-54ad29c6696a admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.596154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bea84e-3934-478e-92b7-54ad29c6696a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.596154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bea84e-3934-478e-92b7-54ad29c6696a admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.596154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bea84e-3934-478e-92b7-54ad29c6696a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.596154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bea84e-3934-478e-92b7-54ad29c6696a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.673216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 791/1585] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:31 2022] POST /identity/v3/users => generated 302 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:31.674178 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828940f-9125-4f8e-95a8-206adef50f8f admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.684221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828940f-9125-4f8e-95a8-206adef50f8f admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.688785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ca1ae-b5b2-4b39-9c54-ff7dc7038229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.689385 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ca1ae-b5b2-4b39-9c54-ff7dc7038229 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.689639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ca1ae-b5b2-4b39-9c54-ff7dc7038229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.689936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ca1ae-b5b2-4b39-9c54-ff7dc7038229 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.690555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ca1ae-b5b2-4b39-9c54-ff7dc7038229 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.692990 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ca1ae-b5b2-4b39-9c54-ff7dc7038229 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.697792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 795/1586] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:31 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:31.697792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bea84e-3934-478e-92b7-54ad29c6696a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.709300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bea84e-3934-478e-92b7-54ad29c6696a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.713807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703773d5-9cc1-4ec0-a500-af3597889bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.714405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703773d5-9cc1-4ec0-a500-af3597889bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.714675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703773d5-9cc1-4ec0-a500-af3597889bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.714941 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703773d5-9cc1-4ec0-a500-af3597889bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.715590 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703773d5-9cc1-4ec0-a500-af3597889bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.717161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703773d5-9cc1-4ec0-a500-af3597889bb4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.726947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 796/1587] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:31.727807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703773d5-9cc1-4ec0-a500-af3597889bb4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.737254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703773d5-9cc1-4ec0-a500-af3597889bb4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.743937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ff7d8-3e66-43de-a7ff-08cfa231b473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.744496 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff7d8-3e66-43de-a7ff-08cfa231b473 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.744773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff7d8-3e66-43de-a7ff-08cfa231b473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.745032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff7d8-3e66-43de-a7ff-08cfa231b473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.745641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ff7d8-3e66-43de-a7ff-08cfa231b473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.747201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ff7d8-3e66-43de-a7ff-08cfa231b473 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.757846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 797/1588] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:31.758720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ff7d8-3e66-43de-a7ff-08cfa231b473 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.767766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ff7d8-3e66-43de-a7ff-08cfa231b473 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.772948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577ff2a-cf04-4fe1-bd9c-61042f227afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.773506 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577ff2a-cf04-4fe1-bd9c-61042f227afe admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.773777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577ff2a-cf04-4fe1-bd9c-61042f227afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.774067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2577ff2a-cf04-4fe1-bd9c-61042f227afe admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.774468 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 792/1589] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:31 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:31.774675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577ff2a-cf04-4fe1-bd9c-61042f227afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.775284 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ca1ae-b5b2-4b39-9c54-ff7dc7038229 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.776609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2577ff2a-cf04-4fe1-bd9c-61042f227afe admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.785414 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ca1ae-b5b2-4b39-9c54-ff7dc7038229 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.790130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ebf5a-453d-40ba-80a8-034c2c23eb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.790739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ebf5a-453d-40ba-80a8-034c2c23eb00 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.791013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ebf5a-453d-40ba-80a8-034c2c23eb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.791273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ebf5a-453d-40ba-80a8-034c2c23eb00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.791900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ebf5a-453d-40ba-80a8-034c2c23eb00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.793495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ebf5a-453d-40ba-80a8-034c2c23eb00 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.804102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 793/1590] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:31.804876 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ebf5a-453d-40ba-80a8-034c2c23eb00 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.857313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ebf5a-453d-40ba-80a8-034c2c23eb00 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.860822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a31c18-1e13-4a7a-9015-f408a4101b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.861417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a31c18-1e13-4a7a-9015-f408a4101b3c service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.861688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a31c18-1e13-4a7a-9015-f408a4101b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.861903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 798/1591] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:31 2022] POST /identity/v3/projects => generated 357 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:31.862021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a31c18-1e13-4a7a-9015-f408a4101b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.862757 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577ff2a-cf04-4fe1-bd9c-61042f227afe admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.872373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a31c18-1e13-4a7a-9015-f408a4101b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.873846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a31c18-1e13-4a7a-9015-f408a4101b3c service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.885538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 794/1592] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:31.886461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a31c18-1e13-4a7a-9015-f408a4101b3c service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.896652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a31c18-1e13-4a7a-9015-f408a4101b3c service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.932475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2577ff2a-cf04-4fe1-bd9c-61042f227afe admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.937437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a4fca-66bb-4366-9bb3-2c2d674695f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.938790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a4fca-66bb-4366-9bb3-2c2d674695f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.938790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a4fca-66bb-4366-9bb3-2c2d674695f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.938790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a4fca-66bb-4366-9bb3-2c2d674695f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.939238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a4fca-66bb-4366-9bb3-2c2d674695f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.940853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a4fca-66bb-4366-9bb3-2c2d674695f5 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.951725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 799/1593] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:31.952600 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a4fca-66bb-4366-9bb3-2c2d674695f5 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.962095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a4fca-66bb-4366-9bb3-2c2d674695f5 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.964874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf780c-14a4-4a7b-9314-30e9a0e44c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.965512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf780c-14a4-4a7b-9314-30e9a0e44c2d admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.965788 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf780c-14a4-4a7b-9314-30e9a0e44c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.966092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf780c-14a4-4a7b-9314-30e9a0e44c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.966725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf780c-14a4-4a7b-9314-30e9a0e44c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.966897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2242c-9220-4aa4-af32-aa626af6c751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.967487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2242c-9220-4aa4-af32-aa626af6c751 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.967765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2242c-9220-4aa4-af32-aa626af6c751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.968031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2242c-9220-4aa4-af32-aa626af6c751 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.968312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf780c-14a4-4a7b-9314-30e9a0e44c2d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.968656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2242c-9220-4aa4-af32-aa626af6c751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:31.970251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2242c-9220-4aa4-af32-aa626af6c751 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:31.979403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 795/1594] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:31.979718 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf780c-14a4-4a7b-9314-30e9a0e44c2d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.985111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 800/1595] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:31.985111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2242c-9220-4aa4-af32-aa626af6c751 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:31.991337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf780c-14a4-4a7b-9314-30e9a0e44c2d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.994254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2242c-9220-4aa4-af32-aa626af6c751 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:31.996558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd865b-a46d-45ce-b6e4-5f4581819c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:31.997173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd865b-a46d-45ce-b6e4-5f4581819c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:31.997519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd865b-a46d-45ce-b6e4-5f4581819c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:31.997859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd865b-a46d-45ce-b6e4-5f4581819c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:31.998375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd865b-a46d-45ce-b6e4-5f4581819c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.002093 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39076d-51ad-420c-82ff-29602f41c035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.002239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd865b-a46d-45ce-b6e4-5f4581819c12 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.002335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39076d-51ad-420c-82ff-29602f41c035 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.002335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39076d-51ad-420c-82ff-29602f41c035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.002335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39076d-51ad-420c-82ff-29602f41c035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.002335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39076d-51ad-420c-82ff-29602f41c035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.004225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39076d-51ad-420c-82ff-29602f41c035 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.006450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:32.014709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 796/1596] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:32.014709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd865b-a46d-45ce-b6e4-5f4581819c12 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:32.017983 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:32.020410 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 801/1597] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:32.020410 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39076d-51ad-420c-82ff-29602f41c035 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:32.026724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd865b-a46d-45ce-b6e4-5f4581819c12 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:32.031137 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39076d-51ad-420c-82ff-29602f41c035 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:32.032285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae4c75-6391-4f78-a71b-23b49d2ea294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.033032 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae4c75-6391-4f78-a71b-23b49d2ea294 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.033396 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae4c75-6391-4f78-a71b-23b49d2ea294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.033743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae4c75-6391-4f78-a71b-23b49d2ea294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.034487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae4c75-6391-4f78-a71b-23b49d2ea294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.036285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f52b5-013d-4265-89ad-39ff4a68f308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.036684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae4c75-6391-4f78-a71b-23b49d2ea294 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.036923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f52b5-013d-4265-89ad-39ff4a68f308 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.037194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f52b5-013d-4265-89ad-39ff4a68f308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.037456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f52b5-013d-4265-89ad-39ff4a68f308 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.038428 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f52b5-013d-4265-89ad-39ff4a68f308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.040018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f52b5-013d-4265-89ad-39ff4a68f308 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.049997 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 797/1598] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:32.052935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae4c75-6391-4f78-a71b-23b49d2ea294 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:32.063333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae4c75-6391-4f78-a71b-23b49d2ea294 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:32.069060 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011033fc-7cb7-4de2-8c3e-55831b2fcaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.071182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011033fc-7cb7-4de2-8c3e-55831b2fcaf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.071182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011033fc-7cb7-4de2-8c3e-55831b2fcaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.071182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011033fc-7cb7-4de2-8c3e-55831b2fcaf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.071182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011033fc-7cb7-4de2-8c3e-55831b2fcaf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.072415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011033fc-7cb7-4de2-8c3e-55831b2fcaf6 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.083325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 798/1599] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:32.086776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011033fc-7cb7-4de2-8c3e-55831b2fcaf6 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:32.096587 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011033fc-7cb7-4de2-8c3e-55831b2fcaf6 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:32.101919 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f4e4e-d5dc-4892-a59c-58f6aa3e58f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.102499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f4e4e-d5dc-4892-a59c-58f6aa3e58f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.102767 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f4e4e-d5dc-4892-a59c-58f6aa3e58f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.103031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f4e4e-d5dc-4892-a59c-58f6aa3e58f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.103666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f4e4e-d5dc-4892-a59c-58f6aa3e58f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.105209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f4e4e-d5dc-4892-a59c-58f6aa3e58f7 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.116113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 799/1600] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:32.116918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f4e4e-d5dc-4892-a59c-58f6aa3e58f7 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:32.126880 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f4e4e-d5dc-4892-a59c-58f6aa3e58f7 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:32.131544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b476cb6-b49d-4cfd-8582-a8819b0ebfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.132154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b476cb6-b49d-4cfd-8582-a8819b0ebfd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.132424 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b476cb6-b49d-4cfd-8582-a8819b0ebfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.132687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b476cb6-b49d-4cfd-8582-a8819b0ebfd8 admin admin] PATH_INFO: `/v3/projects/a8f3274261c64fc59600b14518dd6cc8/users/40659c9a90604b3aac75a7e910bf55c8/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.146747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b476cb6-b49d-4cfd-8582-a8819b0ebfd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f3274261c64fc59600b14518dd6cc8, user_id=40659c9a90604b3aac75a7e910bf55c8, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.147779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: WARNING py.warnings [None req-6b476cb6-b49d-4cfd-8582-a8819b0ebfd8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:42:32.147779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: warnings.warn(msg) Sep 16 05:42:32.147779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: Sep 16 05:42:32.149678 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b476cb6-b49d-4cfd-8582-a8819b0ebfd8 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.170191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 802/1601] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:32 2022] POST /identity/v3/users => generated 285 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:32.171122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f52b5-013d-4265-89ad-39ff4a68f308 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:32.181505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f52b5-013d-4265-89ad-39ff4a68f308 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:32.186268 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1eae4-5125-44fa-ae2c-41547d89ceed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.186909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1eae4-5125-44fa-ae2c-41547d89ceed admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.187183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1eae4-5125-44fa-ae2c-41547d89ceed admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.187451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1eae4-5125-44fa-ae2c-41547d89ceed admin admin] PATH_INFO: `/v3/projects/1a2ef5dbdbb34457963733b3507abb34/users/c205ee9f85924da79b870648698caf64/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.203852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1eae4-5125-44fa-ae2c-41547d89ceed admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a2ef5dbdbb34457963733b3507abb34, user_id=c205ee9f85924da79b870648698caf64, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.206903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1eae4-5125-44fa-ae2c-41547d89ceed admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.260679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 800/1602] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:32 2022] PUT /identity/v3/projects/a8f3274261c64fc59600b14518dd6cc8/users/40659c9a90604b3aac75a7e910bf55c8/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:32.260679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b476cb6-b49d-4cfd-8582-a8819b0ebfd8 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:32.291191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 803/1603] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:32 2022] PUT /identity/v3/projects/1a2ef5dbdbb34457963733b3507abb34/users/c205ee9f85924da79b870648698caf64/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:32.292349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1eae4-5125-44fa-ae2c-41547d89ceed admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:32.337846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b476cb6-b49d-4cfd-8582-a8819b0ebfd8 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:32.358069 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1eae4-5125-44fa-ae2c-41547d89ceed admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:32.362698 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc503c8-d364-4004-be04-1c9de04afbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.363350 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc503c8-d364-4004-be04-1c9de04afbd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.363722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc503c8-d364-4004-be04-1c9de04afbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.364016 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc503c8-d364-4004-be04-1c9de04afbd0 admin admin] PATH_INFO: `/v3/projects/ce6e4d8568944cc78a53f87dec556ec1/users/050a6123ac274eb1a6cf35560cc9d1c0/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.379847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc503c8-d364-4004-be04-1c9de04afbd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce6e4d8568944cc78a53f87dec556ec1, user_id=050a6123ac274eb1a6cf35560cc9d1c0, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.382531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc503c8-d364-4004-be04-1c9de04afbd0 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.397286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0850a-d16d-4d68-823b-a4f392e4bea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.397286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0850a-d16d-4d68-823b-a4f392e4bea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.397491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0850a-d16d-4d68-823b-a4f392e4bea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.397617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0850a-d16d-4d68-823b-a4f392e4bea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.398282 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0850a-d16d-4d68-823b-a4f392e4bea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.399856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0850a-d16d-4d68-823b-a4f392e4bea3 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.410109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 801/1604] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:32.413411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0850a-d16d-4d68-823b-a4f392e4bea3 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:32.423403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0850a-d16d-4d68-823b-a4f392e4bea3 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:32.428247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093be7f-b34f-4ad7-8741-fabbad6ac96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.428894 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093be7f-b34f-4ad7-8741-fabbad6ac96b admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.429162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093be7f-b34f-4ad7-8741-fabbad6ac96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.429425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2093be7f-b34f-4ad7-8741-fabbad6ac96b admin admin] PATH_INFO: `/v3/projects/4012b2fd2ece444686e67cd956d9c651/users/ef4cd8655bfb433db2d118cbd166dc6a/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.445627 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093be7f-b34f-4ad7-8741-fabbad6ac96b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4012b2fd2ece444686e67cd956d9c651, user_id=ef4cd8655bfb433db2d118cbd166dc6a, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.449063 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2093be7f-b34f-4ad7-8741-fabbad6ac96b admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.463451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 804/1605] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:32 2022] PUT /identity/v3/projects/ce6e4d8568944cc78a53f87dec556ec1/users/050a6123ac274eb1a6cf35560cc9d1c0/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:32.463451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc503c8-d364-4004-be04-1c9de04afbd0 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:32.517224 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc503c8-d364-4004-be04-1c9de04afbd0 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:32.530159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acba13-39c0-48ff-9bf3-5a515214c5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.530159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acba13-39c0-48ff-9bf3-5a515214c5cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.530159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acba13-39c0-48ff-9bf3-5a515214c5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.530159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acba13-39c0-48ff-9bf3-5a515214c5cf admin admin] PATH_INFO: `/v3/projects/3cf2362acfe94538bba7793bd690649c/users/edd619bf865645fab0460d3aa847a734/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.541289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82acba13-39c0-48ff-9bf3-5a515214c5cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf2362acfe94538bba7793bd690649c, user_id=edd619bf865645fab0460d3aa847a734, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.545050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82acba13-39c0-48ff-9bf3-5a515214c5cf admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.555123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 802/1606] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:32 2022] PUT /identity/v3/projects/4012b2fd2ece444686e67cd956d9c651/users/ef4cd8655bfb433db2d118cbd166dc6a/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:32.556022 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093be7f-b34f-4ad7-8741-fabbad6ac96b admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:32.589068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:32.623846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2093be7f-b34f-4ad7-8741-fabbad6ac96b admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:32.628962 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbeacc1-898e-4ae8-b63d-18ad1d468ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.630431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeacc1-898e-4ae8-b63d-18ad1d468ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.630840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeacc1-898e-4ae8-b63d-18ad1d468ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.631119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeacc1-898e-4ae8-b63d-18ad1d468ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.631765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbeacc1-898e-4ae8-b63d-18ad1d468ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.633353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbeacc1-898e-4ae8-b63d-18ad1d468ee6 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.647141 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 803/1607] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:32.651018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 805/1608] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:32 2022] PUT /identity/v3/projects/3cf2362acfe94538bba7793bd690649c/users/edd619bf865645fab0460d3aa847a734/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:32.651148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9883b33-8dfd-49d2-b217-162877e48dd3 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.651148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9883b33-8dfd-49d2-b217-162877e48dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.651148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9883b33-8dfd-49d2-b217-162877e48dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.651420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad36fa-bb8d-430e-937b-63e226a07b7e None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.651420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad36fa-bb8d-430e-937b-63e226a07b7e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.651420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad36fa-bb8d-430e-937b-63e226a07b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.668958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-30ad36fa-bb8d-430e-937b-63e226a07b7e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:32.670276 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-30ad36fa-bb8d-430e-937b-63e226a07b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:32.679347 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f9883b33-8dfd-49d2-b217-162877e48dd3 None None] MFA Rules not processed for user `40659c9a90604b3aac75a7e910bf55c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:32.680929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f9883b33-8dfd-49d2-b217-162877e48dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:32.684597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:32.734892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 806/1609] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:32.735733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad36fa-bb8d-430e-937b-63e226a07b7e None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:32.748583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad36fa-bb8d-430e-937b-63e226a07b7e None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:32.753343 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20b28d-aa08-4ddd-b79c-adf8588ebe8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.754022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20b28d-aa08-4ddd-b79c-adf8588ebe8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.754433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20b28d-aa08-4ddd-b79c-adf8588ebe8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.757310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20b28d-aa08-4ddd-b79c-adf8588ebe8c admin admin] PATH_INFO: `/v3/projects/3d0323b2af5040ee976e6a0a223ebe2c/users/8e28d24465df4f319bc07406586789fa/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.771013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20b28d-aa08-4ddd-b79c-adf8588ebe8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d0323b2af5040ee976e6a0a223ebe2c, user_id=8e28d24465df4f319bc07406586789fa, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:32.773705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 804/1610] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:32.773856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20b28d-aa08-4ddd-b79c-adf8588ebe8c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:32.775401 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3d83c-edd7-4bc3-bec2-bf9ea34dc652 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.775858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3d83c-edd7-4bc3-bec2-bf9ea34dc652 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.776283 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3d83c-edd7-4bc3-bec2-bf9ea34dc652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.807483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-ced3d83c-edd7-4bc3-bec2-bf9ea34dc652 None None] MFA Rules not processed for user `050a6123ac274eb1a6cf35560cc9d1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:32.808684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-ced3d83c-edd7-4bc3-bec2-bf9ea34dc652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:32.872878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 805/1611] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:32.874221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafe12e-6d70-4609-9d18-2b57ba0863fa None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.874440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafe12e-6d70-4609-9d18-2b57ba0863fa None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.874706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeafe12e-6d70-4609-9d18-2b57ba0863fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.908452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 807/1612] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:32 2022] PUT /identity/v3/projects/3d0323b2af5040ee976e6a0a223ebe2c/users/8e28d24465df4f319bc07406586789fa/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:32.909381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-eeafe12e-6d70-4609-9d18-2b57ba0863fa None None] MFA Rules not processed for user `ef4cd8655bfb433db2d118cbd166dc6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:32.913854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20b28d-aa08-4ddd-b79c-adf8588ebe8c admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:32.913997 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-eeafe12e-6d70-4609-9d18-2b57ba0863fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:32.946252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:32.984363 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20b28d-aa08-4ddd-b79c-adf8588ebe8c admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:32.988765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 806/1613] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:32.990223 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d946c-c7ab-4791-86cb-ea8e2da790b1 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:32.990514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d946c-c7ab-4791-86cb-ea8e2da790b1 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:32.990853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d946c-c7ab-4791-86cb-ea8e2da790b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:32.993239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb47854-a86d-44e3-a2e6-b551af7d9605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:32.994060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47854-a86d-44e3-a2e6-b551af7d9605 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.005856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47854-a86d-44e3-a2e6-b551af7d9605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.005856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb47854-a86d-44e3-a2e6-b551af7d9605 admin admin] PATH_INFO: `/v3/projects/c0487478220e447fb7ceef03ad440af5/users/e323b30956904754ac6cc03c640c4fc0/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.010562 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f89d946c-c7ab-4791-86cb-ea8e2da790b1 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:33.011703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f89d946c-c7ab-4791-86cb-ea8e2da790b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:33.011848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb47854-a86d-44e3-a2e6-b551af7d9605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0487478220e447fb7ceef03ad440af5, user_id=e323b30956904754ac6cc03c640c4fc0, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:33.014459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb47854-a86d-44e3-a2e6-b551af7d9605 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:33.020017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 807/1614] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:33.023303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d946c-c7ab-4791-86cb-ea8e2da790b1 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:33.074263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d946c-c7ab-4791-86cb-ea8e2da790b1 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:33.078177 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10459a-0190-4212-ad8a-d4d7309ccea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:33.078782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10459a-0190-4212-ad8a-d4d7309ccea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.079058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10459a-0190-4212-ad8a-d4d7309ccea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.079325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10459a-0190-4212-ad8a-d4d7309ccea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.089735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10459a-0190-4212-ad8a-d4d7309ccea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:33.094754 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10459a-0190-4212-ad8a-d4d7309ccea1 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:33.094990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 808/1615] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:32 2022] PUT /identity/v3/projects/c0487478220e447fb7ceef03ad440af5/users/e323b30956904754ac6cc03c640c4fc0/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:33.094990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491b97f-097d-42a5-8513-8667935e0da2 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.094990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491b97f-097d-42a5-8513-8667935e0da2 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.094990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491b97f-097d-42a5-8513-8667935e0da2 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.095358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 809/1616] 10.222.0.49 () {54 vars in 962 bytes} [Fri Sep 16 05:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:42:33.101275 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe98a4b-727a-4298-85a4-554bdcbe66e4 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.101557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe98a4b-727a-4298-85a4-554bdcbe66e4 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.101854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe98a4b-727a-4298-85a4-554bdcbe66e4 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.102197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 808/1617] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:33.103111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 810/1618] 10.222.0.49 () {54 vars in 962 bytes} [Fri Sep 16 05:42:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:42:33.103513 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673eea0-059a-4e8e-991c-69c76cd7472d None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.103784 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673eea0-059a-4e8e-991c-69c76cd7472d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.103913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe98a4b-727a-4298-85a4-554bdcbe66e4 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:33.104051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673eea0-059a-4e8e-991c-69c76cd7472d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.124813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-2673eea0-059a-4e8e-991c-69c76cd7472d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:33.126052 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2673eea0-059a-4e8e-991c-69c76cd7472d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:33.187047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe98a4b-727a-4298-85a4-554bdcbe66e4 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:33.191984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7ca7e-c496-42ca-b05b-ac80a4ebb2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:33.193137 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7ca7e-c496-42ca-b05b-ac80a4ebb2ef admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.193137 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7ca7e-c496-42ca-b05b-ac80a4ebb2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.193388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf7ca7e-c496-42ca-b05b-ac80a4ebb2ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.193846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7ca7e-c496-42ca-b05b-ac80a4ebb2ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:33.195901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf7ca7e-c496-42ca-b05b-ac80a4ebb2ef admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:33.200306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 809/1619] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:33.201416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673eea0-059a-4e8e-991c-69c76cd7472d None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:33.280794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673eea0-059a-4e8e-991c-69c76cd7472d None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:33.285561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a25e5e9-9dd4-47fd-814d-32ea62be116c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:33.286211 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25e5e9-9dd4-47fd-814d-32ea62be116c service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.286486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25e5e9-9dd4-47fd-814d-32ea62be116c service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.286752 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a25e5e9-9dd4-47fd-814d-32ea62be116c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.298233 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25e5e9-9dd4-47fd-814d-32ea62be116c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:33.300869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a25e5e9-9dd4-47fd-814d-32ea62be116c service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:33.303413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 811/1620] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:33 2022] POST /identity/v3/projects => generated 403 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:33.303824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7ca7e-c496-42ca-b05b-ac80a4ebb2ef admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:33.309964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:33.316486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 810/1621] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:33.317930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a95399-c050-4678-b58f-619d4ed747b9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.318553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a95399-c050-4678-b58f-619d4ed747b9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.318831 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a95399-c050-4678-b58f-619d4ed747b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.362716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-73a95399-c050-4678-b58f-619d4ed747b9 None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:33.364369 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-73a95399-c050-4678-b58f-619d4ed747b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:33.376149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf7ca7e-c496-42ca-b05b-ac80a4ebb2ef admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:33.386213 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e01d51-2c23-4313-b7e4-a7e9adc490f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:33.387032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e01d51-2c23-4313-b7e4-a7e9adc490f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.387411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e01d51-2c23-4313-b7e4-a7e9adc490f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.387751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e01d51-2c23-4313-b7e4-a7e9adc490f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.388535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e01d51-2c23-4313-b7e4-a7e9adc490f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:33.390620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e01d51-2c23-4313-b7e4-a7e9adc490f3 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:33.399351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:33.410176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 812/1622] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:33.410176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36fad8-fddc-425f-ac57-924006dbe429 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.410176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36fad8-fddc-425f-ac57-924006dbe429 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.410176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36fad8-fddc-425f-ac57-924006dbe429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.430474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-4c36fad8-fddc-425f-ac57-924006dbe429 None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:33.431731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4c36fad8-fddc-425f-ac57-924006dbe429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:33.459701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 811/1623] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Sep 16 05:42:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:33.460739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a95399-c050-4678-b58f-619d4ed747b9 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:33.462403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-73a95399-c050-4678-b58f-619d4ed747b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:33.530685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a95399-c050-4678-b58f-619d4ed747b9 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:33.534473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629febb-af50-4918-93d0-683568db57c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:33.535119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629febb-af50-4918-93d0-683568db57c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.535413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629febb-af50-4918-93d0-683568db57c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.535842 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629febb-af50-4918-93d0-683568db57c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.545688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 813/1624] 10.222.0.49 () {58 vars in 1059 bytes} [Fri Sep 16 05:42:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:33.546612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36fad8-fddc-425f-ac57-924006dbe429 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:33.557251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629febb-af50-4918-93d0-683568db57c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:33.558840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a629febb-af50-4918-93d0-683568db57c0 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:33.560934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36fad8-fddc-425f-ac57-924006dbe429 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:33.566942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77238c3b-7ff8-4b2e-9080-63a151c43347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:33.567570 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77238c3b-7ff8-4b2e-9080-63a151c43347 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.567849 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77238c3b-7ff8-4b2e-9080-63a151c43347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.568120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77238c3b-7ff8-4b2e-9080-63a151c43347 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.569266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77238c3b-7ff8-4b2e-9080-63a151c43347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:33.571788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77238c3b-7ff8-4b2e-9080-63a151c43347 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:33.582440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 812/1625] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:33.584239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629febb-af50-4918-93d0-683568db57c0 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:33.605433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629febb-af50-4918-93d0-683568db57c0 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:33.617136 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca95879-5118-460a-92e7-8b7645923bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:33.618429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca95879-5118-460a-92e7-8b7645923bea admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.618722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca95879-5118-460a-92e7-8b7645923bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.619375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca95879-5118-460a-92e7-8b7645923bea admin admin] PATH_INFO: `/v3/projects/c0487478220e447fb7ceef03ad440af5/users/e323b30956904754ac6cc03c640c4fc0/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.622972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca95879-5118-460a-92e7-8b7645923bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0487478220e447fb7ceef03ad440af5, user_id=e323b30956904754ac6cc03c640c4fc0, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:33.630543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca95879-5118-460a-92e7-8b7645923bea admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:33.642859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 813/1626] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:33 2022] PUT /identity/v3/projects/c0487478220e447fb7ceef03ad440af5/users/e323b30956904754ac6cc03c640c4fc0/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:33.643758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca95879-5118-460a-92e7-8b7645923bea admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:33.691110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 814/1627] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:33 2022] POST /identity/v3/users => generated 308 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:33.693084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77238c3b-7ff8-4b2e-9080-63a151c43347 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:33.707108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77238c3b-7ff8-4b2e-9080-63a151c43347 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:33.738082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca95879-5118-460a-92e7-8b7645923bea admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:33.751383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ae1fc-855b-4125-8fc7-f4ef1c8bc017 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:33.752013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ae1fc-855b-4125-8fc7-f4ef1c8bc017 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.752424 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ae1fc-855b-4125-8fc7-f4ef1c8bc017 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.753236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ae1fc-855b-4125-8fc7-f4ef1c8bc017 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.787606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34e97f-67b0-4c3e-afc2-ea04826ede0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:33.788559 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34e97f-67b0-4c3e-afc2-ea04826ede0a service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.788846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34e97f-67b0-4c3e-afc2-ea04826ede0a service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.789125 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34e97f-67b0-4c3e-afc2-ea04826ede0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.792859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:33.804777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:33.901874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ae1fc-855b-4125-8fc7-f4ef1c8bc017 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:33.903523 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ae1fc-855b-4125-8fc7-f4ef1c8bc017 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:33.920539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34e97f-67b0-4c3e-afc2-ea04826ede0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:33.927704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34e97f-67b0-4c3e-afc2-ea04826ede0a service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:33.936105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 814/1628] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:33.939711 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 815/1629] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:33.943714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf2593-c538-49ad-856d-e90874a7cb3f None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:33.946918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf2593-c538-49ad-856d-e90874a7cb3f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:33.950363 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf2593-c538-49ad-856d-e90874a7cb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:33.958618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34e97f-67b0-4c3e-afc2-ea04826ede0a service nova] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:34.034588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-5bbf2593-c538-49ad-856d-e90874a7cb3f None None] MFA Rules not processed for user `e323b30956904754ac6cc03c640c4fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:34.036939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5bbf2593-c538-49ad-856d-e90874a7cb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:34.120033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34e97f-67b0-4c3e-afc2-ea04826ede0a service nova] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:34.127100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41063b99-98f5-4076-a79e-cceab1a80a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:34.127919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41063b99-98f5-4076-a79e-cceab1a80a58 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:34.128355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41063b99-98f5-4076-a79e-cceab1a80a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:34.128716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41063b99-98f5-4076-a79e-cceab1a80a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:34.129863 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41063b99-98f5-4076-a79e-cceab1a80a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:34.136655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41063b99-98f5-4076-a79e-cceab1a80a58 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:34.169327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 816/1630] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:33 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:34.171275 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41063b99-98f5-4076-a79e-cceab1a80a58 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:34.185721 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 815/1631] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:33 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:34.201156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf2593-c538-49ad-856d-e90874a7cb3f None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:34.214683 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41063b99-98f5-4076-a79e-cceab1a80a58 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:34.231403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36a937-0f7e-44a3-97fc-369bc6a48cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:34.233061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36a937-0f7e-44a3-97fc-369bc6a48cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:34.233390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36a937-0f7e-44a3-97fc-369bc6a48cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:34.237552 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36a937-0f7e-44a3-97fc-369bc6a48cfe admin admin] PATH_INFO: `/v3/projects/808314251d5b4d38b40f4712effb03d7/users/84ecb1e127f34713a3023054c8817c71/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:34.275289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36a937-0f7e-44a3-97fc-369bc6a48cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=808314251d5b4d38b40f4712effb03d7, user_id=84ecb1e127f34713a3023054c8817c71, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:34.283795 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36a937-0f7e-44a3-97fc-369bc6a48cfe admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:34.324560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf2593-c538-49ad-856d-e90874a7cb3f None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:34.329180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b388565-d1c3-4a7f-98c9-8b485a4303b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:34.329803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b388565-d1c3-4a7f-98c9-8b485a4303b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:34.330151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b388565-d1c3-4a7f-98c9-8b485a4303b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:34.330551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b388565-d1c3-4a7f-98c9-8b485a4303b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:34.345703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b388565-d1c3-4a7f-98c9-8b485a4303b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:34.348704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b388565-d1c3-4a7f-98c9-8b485a4303b5 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:34.381143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 816/1632] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:34 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:34.406395 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b388565-d1c3-4a7f-98c9-8b485a4303b5 service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:34.411177 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 817/1633] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:34 2022] PUT /identity/v3/projects/808314251d5b4d38b40f4712effb03d7/users/84ecb1e127f34713a3023054c8817c71/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:34.433182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b388565-d1c3-4a7f-98c9-8b485a4303b5 service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:34.445988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1526bbd-ea16-467a-aec2-fec9b9a23d9a None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:34.446342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1526bbd-ea16-467a-aec2-fec9b9a23d9a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:34.446689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1526bbd-ea16-467a-aec2-fec9b9a23d9a None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:34.448795 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 818/1634] 10.222.0.49 () {54 vars in 912 bytes} [Fri Sep 16 05:42:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:42:34.454586 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545c198-8202-4ced-9199-0f1a9370bad5 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:34.454746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545c198-8202-4ced-9199-0f1a9370bad5 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:34.456790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545c198-8202-4ced-9199-0f1a9370bad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:34.492908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-3545c198-8202-4ced-9199-0f1a9370bad5 None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:34.500055 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3545c198-8202-4ced-9199-0f1a9370bad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:34.540357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:34.602023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9735c4-3e58-4348-9f83-a4ffc1cf24c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:34.604760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9735c4-3e58-4348-9f83-a4ffc1cf24c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:34.608129 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9735c4-3e58-4348-9f83-a4ffc1cf24c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:34.608129 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9735c4-3e58-4348-9f83-a4ffc1cf24c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:34.608129 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9735c4-3e58-4348-9f83-a4ffc1cf24c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:34.611036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9735c4-3e58-4348-9f83-a4ffc1cf24c4 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:34.631679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 817/1635] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:34.631862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 819/1636] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:42:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:42:34.633949 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545c198-8202-4ced-9199-0f1a9370bad5 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:34.644998 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6099190-908d-4bbf-be06-608d4eeeb34b None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:34.645644 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6099190-908d-4bbf-be06-608d4eeeb34b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:34.646418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6099190-908d-4bbf-be06-608d4eeeb34b None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:34.654849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 818/1637] 10.222.0.49 () {54 vars in 912 bytes} [Fri Sep 16 05:42:34 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:42:34.656109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6099190-908d-4bbf-be06-608d4eeeb34b None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:34.673780 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545c198-8202-4ced-9199-0f1a9370bad5 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:34.680775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610ec4f-302c-4590-b856-32e3f5d069c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:34.685799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610ec4f-302c-4590-b856-32e3f5d069c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:34.685799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610ec4f-302c-4590-b856-32e3f5d069c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:34.685799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610ec4f-302c-4590-b856-32e3f5d069c6 admin admin] PATH_INFO: `/v3/projects/808314251d5b4d38b40f4712effb03d7/users/84ecb1e127f34713a3023054c8817c71/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:34.685799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610ec4f-302c-4590-b856-32e3f5d069c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=808314251d5b4d38b40f4712effb03d7, user_id=84ecb1e127f34713a3023054c8817c71, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:34.687814 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610ec4f-302c-4590-b856-32e3f5d069c6 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:34.703584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 820/1638] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:34 2022] PUT /identity/v3/projects/808314251d5b4d38b40f4712effb03d7/users/84ecb1e127f34713a3023054c8817c71/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:42:34.715118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d96517-0aec-48bd-be17-ca614657052f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:34.719220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d96517-0aec-48bd-be17-ca614657052f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:34.719220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d96517-0aec-48bd-be17-ca614657052f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:34.725644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:34.741584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-14d96517-0aec-48bd-be17-ca614657052f None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:34.743464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-14d96517-0aec-48bd-be17-ca614657052f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:35.012638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6099190-908d-4bbf-be06-608d4eeeb34b None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:35.017424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 821/1639] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:42:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:35.017424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d96517-0aec-48bd-be17-ca614657052f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:35.018525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1bc55-6fb1-4046-a369-4781567628ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:35.018525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1bc55-6fb1-4046-a369-4781567628ad service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:35.018525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1bc55-6fb1-4046-a369-4781567628ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:35.018525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1bc55-6fb1-4046-a369-4781567628ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:35.031854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1bc55-6fb1-4046-a369-4781567628ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:35.031854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1bc55-6fb1-4046-a369-4781567628ad service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:35.031854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 819/1640] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:35.031854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2463be-57a3-4750-b45a-4b4083884b85 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:35.031854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2463be-57a3-4750-b45a-4b4083884b85 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:35.031854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2463be-57a3-4750-b45a-4b4083884b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:35.046903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d96517-0aec-48bd-be17-ca614657052f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:35.055933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f390410-bfcd-47d3-a2bb-f02a70e3611a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:35.056567 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f390410-bfcd-47d3-a2bb-f02a70e3611a service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:35.057787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-9e2463be-57a3-4750-b45a-4b4083884b85 None None] MFA Rules not processed for user `84ecb1e127f34713a3023054c8817c71`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:35.058550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f390410-bfcd-47d3-a2bb-f02a70e3611a service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:35.058849 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f390410-bfcd-47d3-a2bb-f02a70e3611a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:35.060665 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9e2463be-57a3-4750-b45a-4b4083884b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:35.202349 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 820/1641] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:34 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:35.216840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f390410-bfcd-47d3-a2bb-f02a70e3611a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:35.218036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f390410-bfcd-47d3-a2bb-f02a70e3611a service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:35.245561 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 822/1642] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:35.970703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2463be-57a3-4750-b45a-4b4083884b85 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:35.983010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2463be-57a3-4750-b45a-4b4083884b85 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:35.992844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa8ff8-ffb0-4fb6-91b8-43344c3ac080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:35.999966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa8ff8-ffb0-4fb6-91b8-43344c3ac080 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:35.999966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa8ff8-ffb0-4fb6-91b8-43344c3ac080 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:35.999966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa8ff8-ffb0-4fb6-91b8-43344c3ac080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:36.010184 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa8ff8-ffb0-4fb6-91b8-43344c3ac080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:36.012158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa8ff8-ffb0-4fb6-91b8-43344c3ac080 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:36.055162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 821/1643] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:36.189607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f390410-bfcd-47d3-a2bb-f02a70e3611a service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:36.192836 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f390410-bfcd-47d3-a2bb-f02a70e3611a service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:36.209323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f037409-42f3-4e62-b122-65829001d3a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:36.211059 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f037409-42f3-4e62-b122-65829001d3a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:36.217194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f037409-42f3-4e62-b122-65829001d3a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:36.217194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f037409-42f3-4e62-b122-65829001d3a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:36.237081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f037409-42f3-4e62-b122-65829001d3a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:36.239871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f037409-42f3-4e62-b122-65829001d3a5 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:36.266152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 823/1644] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:36 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:36.561757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa8ff8-ffb0-4fb6-91b8-43344c3ac080 service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:36.574048 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:36.682062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa8ff8-ffb0-4fb6-91b8-43344c3ac080 service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:36.693672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b85879-9760-4b11-bcca-8d18b678bf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:36.694783 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b85879-9760-4b11-bcca-8d18b678bf3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:36.695058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b85879-9760-4b11-bcca-8d18b678bf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:36.695325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b85879-9760-4b11-bcca-8d18b678bf3f admin admin] PATH_INFO: `/v3/users/40659c9a90604b3aac75a7e910bf55c8` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:36.701873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b85879-9760-4b11-bcca-8d18b678bf3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=40659c9a90604b3aac75a7e910bf55c8)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:36.702925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: WARNING py.warnings [None req-96b85879-9760-4b11-bcca-8d18b678bf3f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:42:36.702925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: warnings.warn(msg) Sep 16 05:42:36.702925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: Sep 16 05:42:36.706943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b85879-9760-4b11-bcca-8d18b678bf3f admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:36.789479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f037409-42f3-4e62-b122-65829001d3a5 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:36.804797 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f037409-42f3-4e62-b122-65829001d3a5 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:36.829021 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed7104-384f-403b-a30d-f9d5c56c9180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:36.830209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed7104-384f-403b-a30d-f9d5c56c9180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:36.835427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed7104-384f-403b-a30d-f9d5c56c9180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:36.835427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed7104-384f-403b-a30d-f9d5c56c9180 admin admin] PATH_INFO: `/v3/users/ef4cd8655bfb433db2d118cbd166dc6a` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:36.835427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed7104-384f-403b-a30d-f9d5c56c9180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef4cd8655bfb433db2d118cbd166dc6a)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:36.835427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed7104-384f-403b-a30d-f9d5c56c9180 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:37.092125 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:37.102258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-96b85879-9760-4b11-bcca-8d18b678bf3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40659c9a90604b3aac75a7e910bf55c8'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:37.103496 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-96b85879-9760-4b11-bcca-8d18b678bf3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40659c9a90604b3aac75a7e910bf55c8'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:37.127391 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f3ed7104-384f-403b-a30d-f9d5c56c9180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef4cd8655bfb433db2d118cbd166dc6a'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:37.128155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f3ed7104-384f-403b-a30d-f9d5c56c9180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef4cd8655bfb433db2d118cbd166dc6a'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:37.177791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-96b85879-9760-4b11-bcca-8d18b678bf3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40659c9a90604b3aac75a7e910bf55c8'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:37.200738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-96b85879-9760-4b11-bcca-8d18b678bf3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40659c9a90604b3aac75a7e910bf55c8'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:37.201632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-96b85879-9760-4b11-bcca-8d18b678bf3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40659c9a90604b3aac75a7e910bf55c8'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:37.267890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 822/1645] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:42:36 2022] DELETE /identity/v3/users/40659c9a90604b3aac75a7e910bf55c8 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:37.276662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f3ed7104-384f-403b-a30d-f9d5c56c9180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef4cd8655bfb433db2d118cbd166dc6a'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:37.278897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f3ed7104-384f-403b-a30d-f9d5c56c9180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef4cd8655bfb433db2d118cbd166dc6a'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:37.296454 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f3ed7104-384f-403b-a30d-f9d5c56c9180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef4cd8655bfb433db2d118cbd166dc6a'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:37.297690 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fdeda-9b34-453e-90e8-50f2650d5a49 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:37.300478 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fdeda-9b34-453e-90e8-50f2650d5a49 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:37.300807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fdeda-9b34-453e-90e8-50f2650d5a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:37.330092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 824/1646] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:42:36 2022] DELETE /identity/v3/users/ef4cd8655bfb433db2d118cbd166dc6a => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:37.338208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e5514-810f-4af1-b565-74595ffedb23 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:37.338995 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e5514-810f-4af1-b565-74595ffedb23 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:37.339408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e5514-810f-4af1-b565-74595ffedb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:37.371966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a53e5514-810f-4af1-b565-74595ffedb23 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:37.375907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a53e5514-810f-4af1-b565-74595ffedb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:37.382593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-eb5fdeda-9b34-453e-90e8-50f2650d5a49 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:37.384518 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-eb5fdeda-9b34-453e-90e8-50f2650d5a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:37.538477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 823/1647] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:37.545873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 825/1648] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:37.567595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fdeda-9b34-453e-90e8-50f2650d5a49 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:37.568173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-eb5fdeda-9b34-453e-90e8-50f2650d5a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:37.570258 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e5514-810f-4af1-b565-74595ffedb23 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:37.574036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a53e5514-810f-4af1-b565-74595ffedb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:37.628244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:37.685321 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e5514-810f-4af1-b565-74595ffedb23 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:37.689267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fdeda-9b34-453e-90e8-50f2650d5a49 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:37.689868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499e3e7-9c0d-43f0-9679-f2a589618b2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:37.690573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499e3e7-9c0d-43f0-9679-f2a589618b2d service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:37.690940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499e3e7-9c0d-43f0-9679-f2a589618b2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:37.691213 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499e3e7-9c0d-43f0-9679-f2a589618b2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:37.697267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c11c8-ca04-49e3-b395-5e245f227c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:37.698247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c11c8-ca04-49e3-b395-5e245f227c6b service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:37.698716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c11c8-ca04-49e3-b395-5e245f227c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:37.699171 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c11c8-ca04-49e3-b395-5e245f227c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:37.704455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499e3e7-9c0d-43f0-9679-f2a589618b2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:37.705939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3499e3e7-9c0d-43f0-9679-f2a589618b2d service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:37.712810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c11c8-ca04-49e3-b395-5e245f227c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:37.715051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c11c8-ca04-49e3-b395-5e245f227c6b service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:37.718583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 826/1649] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:37.749120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 824/1650] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:39.229309 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499e3e7-9c0d-43f0-9679-f2a589618b2d service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:39.231474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3499e3e7-9c0d-43f0-9679-f2a589618b2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:39.264376 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c11c8-ca04-49e3-b395-5e245f227c6b service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:39.266429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e68c11c8-ca04-49e3-b395-5e245f227c6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:39.289076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c11c8-ca04-49e3-b395-5e245f227c6b service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:39.301974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b62e11-bc45-4a34-b804-ea8e084d0667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:39.304685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b62e11-bc45-4a34-b804-ea8e084d0667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:39.305108 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b62e11-bc45-4a34-b804-ea8e084d0667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:39.306772 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b62e11-bc45-4a34-b804-ea8e084d0667 admin admin] PATH_INFO: `/v3/projects/a8f3274261c64fc59600b14518dd6cc8` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:39.308261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499e3e7-9c0d-43f0-9679-f2a589618b2d service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:39.310242 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b62e11-bc45-4a34-b804-ea8e084d0667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8f3274261c64fc59600b14518dd6cc8)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:39.314722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b62e11-bc45-4a34-b804-ea8e084d0667 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:39.330453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:39.333306 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:39.334041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:39.334428 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] PATH_INFO: `/v3/projects/4012b2fd2ece444686e67cd956d9c651` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:39.354556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4012b2fd2ece444686e67cd956d9c651)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:39.356078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: WARNING py.warnings [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 16 05:42:39.356078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: warnings.warn(msg) Sep 16 05:42:39.356078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: Sep 16 05:42:39.362415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:39.476696 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-57b62e11-bc45-4a34-b804-ea8e084d0667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8f3274261c64fc59600b14518dd6cc8', 'request_id': 'req-57b62e11-bc45-4a34-b804-ea8e084d0667'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:39.500783 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-57b62e11-bc45-4a34-b804-ea8e084d0667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8f3274261c64fc59600b14518dd6cc8', 'request_id': 'req-57b62e11-bc45-4a34-b804-ea8e084d0667'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:39.524526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-57b62e11-bc45-4a34-b804-ea8e084d0667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8f3274261c64fc59600b14518dd6cc8', 'request_id': 'req-57b62e11-bc45-4a34-b804-ea8e084d0667'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:39.642630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:39.656938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4012b2fd2ece444686e67cd956d9c651', 'request_id': 'req-e909bb2f-d31a-4d03-b52a-efb289f90314'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:39.690751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-57b62e11-bc45-4a34-b804-ea8e084d0667 admin admin] The token cache is being invalidate because project a8f3274261c64fc59600b14518dd6cc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:42:39.691360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-57b62e11-bc45-4a34-b804-ea8e084d0667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:39.691978 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4012b2fd2ece444686e67cd956d9c651', 'request_id': 'req-e909bb2f-d31a-4d03-b52a-efb289f90314'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:39.698159 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-57b62e11-bc45-4a34-b804-ea8e084d0667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:39.700984 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 825/1651] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:42:39 2022] DELETE /identity/v3/projects/a8f3274261c64fc59600b14518dd6cc8 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:39.708871 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd022b-bfc6-45b7-b126-f9dd437438e8 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:39.709389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd022b-bfc6-45b7-b126-f9dd437438e8 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:39.710409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd022b-bfc6-45b7-b126-f9dd437438e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:39.740824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4012b2fd2ece444686e67cd956d9c651', 'request_id': 'req-e909bb2f-d31a-4d03-b52a-efb289f90314'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:39.835666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-18cd022b-bfc6-45b7-b126-f9dd437438e8 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:39.836930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-18cd022b-bfc6-45b7-b126-f9dd437438e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:39.886446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] The token cache is being invalidate because project 4012b2fd2ece444686e67cd956d9c651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:42:39.886628 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:39.887259 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:39.891773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 827/1652] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:42:39 2022] DELETE /identity/v3/projects/4012b2fd2ece444686e67cd956d9c651 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:39.895106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:39.905527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:40.112263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 826/1653] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:42:40.114581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e714a0-6101-49c6-8554-84f5e820bc16 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:40.127337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e714a0-6101-49c6-8554-84f5e820bc16 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:40.127869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e714a0-6101-49c6-8554-84f5e820bc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:40.190766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-85e714a0-6101-49c6-8554-84f5e820bc16 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:40.192138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-85e714a0-6101-49c6-8554-84f5e820bc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:40.248811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 827/1654] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:40.261601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e714a0-6101-49c6-8554-84f5e820bc16 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:40.273389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e714a0-6101-49c6-8554-84f5e820bc16 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:40.275686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e909bb2f-d31a-4d03-b52a-efb289f90314 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:40.280274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcc82b-1fbb-4619-aade-ec4b82ff0b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:40.281098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcc82b-1fbb-4619-aade-ec4b82ff0b1c admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:40.281509 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcc82b-1fbb-4619-aade-ec4b82ff0b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:40.281944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dcc82b-1fbb-4619-aade-ec4b82ff0b1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:40.282918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcc82b-1fbb-4619-aade-ec4b82ff0b1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:42:40.283356 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcc82b-1fbb-4619-aade-ec4b82ff0b1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:40.285162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dcc82b-1fbb-4619-aade-ec4b82ff0b1c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:40.290969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:40.300618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:40.300618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:40.300618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] PATH_INFO: `/v3/users/84ecb1e127f34713a3023054c8817c71` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:40.300618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=84ecb1e127f34713a3023054c8817c71)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:40.305694 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:40.314846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 828/1655] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:42:40.315782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcc82b-1fbb-4619-aade-ec4b82ff0b1c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:40.343528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dcc82b-1fbb-4619-aade-ec4b82ff0b1c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:40.358432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f2f01-1c58-47ee-9fa7-c755024797a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:40.359059 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f2f01-1c58-47ee-9fa7-c755024797a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:40.359538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f2f01-1c58-47ee-9fa7-c755024797a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:40.359809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f2f01-1c58-47ee-9fa7-c755024797a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:40.363730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f2f01-1c58-47ee-9fa7-c755024797a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:40.365865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f2f01-1c58-47ee-9fa7-c755024797a1 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:40.491891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 829/1656] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:40 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:40.492932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f2f01-1c58-47ee-9fa7-c755024797a1 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:40.523304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:40.539746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84ecb1e127f34713a3023054c8817c71'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:40.540451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84ecb1e127f34713a3023054c8817c71'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:40.599885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84ecb1e127f34713a3023054c8817c71'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:40.600657 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84ecb1e127f34713a3023054c8817c71'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:40.616808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f2f01-1c58-47ee-9fa7-c755024797a1 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:40.621086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84ecb1e127f34713a3023054c8817c71'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:40.649461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423170d-7806-436e-ba94-9277b4d60c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:40.649461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423170d-7806-436e-ba94-9277b4d60c7b admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:40.649461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423170d-7806-436e-ba94-9277b4d60c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:40.649461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423170d-7806-436e-ba94-9277b4d60c7b admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:40.650066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423170d-7806-436e-ba94-9277b4d60c7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:40.662591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423170d-7806-436e-ba94-9277b4d60c7b admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:40.683695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 828/1657] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:42:39 2022] DELETE /identity/v3/users/84ecb1e127f34713a3023054c8817c71 => generated 0 bytes in 789 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:40.831819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 830/1658] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:40 2022] POST /identity/v3/users => generated 290 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:40.834084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:40.835657 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:40.953328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:40.983776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7728e5-a669-4a0b-acdd-76ee4ecf0d1f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:40.998152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3613953-0ebc-4cad-9036-dd8eaf155d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:40.999063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3613953-0ebc-4cad-9036-dd8eaf155d5f admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:40.999437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3613953-0ebc-4cad-9036-dd8eaf155d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:40.999801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3613953-0ebc-4cad-9036-dd8eaf155d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:41.000575 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3613953-0ebc-4cad-9036-dd8eaf155d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:41.004683 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3613953-0ebc-4cad-9036-dd8eaf155d5f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:41.028799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 829/1659] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:41.032561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423170d-7806-436e-ba94-9277b4d60c7b admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:41.046522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423170d-7806-436e-ba94-9277b4d60c7b admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:41.058387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec7d1fa-dfb1-4b44-9445-b162b3dfea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:41.059412 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7d1fa-dfb1-4b44-9445-b162b3dfea07 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:41.059412 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7d1fa-dfb1-4b44-9445-b162b3dfea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:41.059887 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec7d1fa-dfb1-4b44-9445-b162b3dfea07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:41.060857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7d1fa-dfb1-4b44-9445-b162b3dfea07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:41.062772 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec7d1fa-dfb1-4b44-9445-b162b3dfea07 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:41.080083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3613953-0ebc-4cad-9036-dd8eaf155d5f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:41.081300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 831/1660] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:41.100730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3613953-0ebc-4cad-9036-dd8eaf155d5f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:41.115715 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69a7cc-ee0c-489f-afc4-b7e251b396f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:41.117277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69a7cc-ee0c-489f-afc4-b7e251b396f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:41.117277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69a7cc-ee0c-489f-afc4-b7e251b396f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:41.117277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69a7cc-ee0c-489f-afc4-b7e251b396f2 admin admin] PATH_INFO: `/v3/projects/0f4504fc10b84d0aa5a5fd253a89f54f/users/c945ac8de66e4807bd82e0a63f20e3d5/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:41.154154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69a7cc-ee0c-489f-afc4-b7e251b396f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f4504fc10b84d0aa5a5fd253a89f54f, user_id=c945ac8de66e4807bd82e0a63f20e3d5, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:41.158422 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69a7cc-ee0c-489f-afc4-b7e251b396f2 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:41.290758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 830/1661] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:41 2022] PUT /identity/v3/projects/0f4504fc10b84d0aa5a5fd253a89f54f/users/c945ac8de66e4807bd82e0a63f20e3d5/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:41.292774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a316bd6-233e-47cb-8c8f-81188b203ecf None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:41.293181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a316bd6-233e-47cb-8c8f-81188b203ecf None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:41.293522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a316bd6-233e-47cb-8c8f-81188b203ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:41.398843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-1a316bd6-233e-47cb-8c8f-81188b203ecf None None] MFA Rules not processed for user `c945ac8de66e4807bd82e0a63f20e3d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:41.402756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1a316bd6-233e-47cb-8c8f-81188b203ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:41.576217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 832/1662] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:41.581178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69a7cc-ee0c-489f-afc4-b7e251b396f2 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:41.734767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69a7cc-ee0c-489f-afc4-b7e251b396f2 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:41.751505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d1365-db84-403b-94f4-52da8328b728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:41.752578 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1365-db84-403b-94f4-52da8328b728 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:41.753121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1365-db84-403b-94f4-52da8328b728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:41.753515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1365-db84-403b-94f4-52da8328b728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:41.754597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d1365-db84-403b-94f4-52da8328b728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:41.757451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d1365-db84-403b-94f4-52da8328b728 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:41.881152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 831/1663] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:41 2022] POST /identity/v3/projects => generated 379 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:42:41.886019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a316bd6-233e-47cb-8c8f-81188b203ecf None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:41.901939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a316bd6-233e-47cb-8c8f-81188b203ecf None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:41.999727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3b7d4-2a93-4774-8c63-c4ba6a896efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:42.000683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3b7d4-2a93-4774-8c63-c4ba6a896efb admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:42.001113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3b7d4-2a93-4774-8c63-c4ba6a896efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:42.001753 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3b7d4-2a93-4774-8c63-c4ba6a896efb admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:42.002570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3b7d4-2a93-4774-8c63-c4ba6a896efb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:42.005889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3b7d4-2a93-4774-8c63-c4ba6a896efb admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:42.161776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 833/1664] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:41 2022] POST /identity/v3/users => generated 290 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:42.165414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d1365-db84-403b-94f4-52da8328b728 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:42.181996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d1365-db84-403b-94f4-52da8328b728 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:42.192497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fab8ad-332b-48c2-a0fb-3ba3e1fa6b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:42.196092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fab8ad-332b-48c2-a0fb-3ba3e1fa6b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:42.196383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fab8ad-332b-48c2-a0fb-3ba3e1fa6b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:42.196654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fab8ad-332b-48c2-a0fb-3ba3e1fa6b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:42.199341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fab8ad-332b-48c2-a0fb-3ba3e1fa6b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:42.202805 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fab8ad-332b-48c2-a0fb-3ba3e1fa6b7d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:42.205058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3b7d4-2a93-4774-8c63-c4ba6a896efb admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:42.207758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-92b3b7d4-2a93-4774-8c63-c4ba6a896efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:42.246515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3b7d4-2a93-4774-8c63-c4ba6a896efb admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:42.251698 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 832/1665] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:42.255985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fab8ad-332b-48c2-a0fb-3ba3e1fa6b7d admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:42.264172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ada1a-8e87-43ee-b921-08fa1fcd3878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:42.266010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ada1a-8e87-43ee-b921-08fa1fcd3878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:42.266353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ada1a-8e87-43ee-b921-08fa1fcd3878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:42.266851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ada1a-8e87-43ee-b921-08fa1fcd3878 admin admin] PATH_INFO: `/v3/projects/808314251d5b4d38b40f4712effb03d7` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:42.269170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ada1a-8e87-43ee-b921-08fa1fcd3878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=808314251d5b4d38b40f4712effb03d7)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:42.271226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fab8ad-332b-48c2-a0fb-3ba3e1fa6b7d admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:42.275634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ada1a-8e87-43ee-b921-08fa1fcd3878 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:42.279811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d4441-8663-4974-9097-9f493ab9fe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:42.283347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d4441-8663-4974-9097-9f493ab9fe36 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:42.284085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d4441-8663-4974-9097-9f493ab9fe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:42.285580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d4441-8663-4974-9097-9f493ab9fe36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:42.289148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d4441-8663-4974-9097-9f493ab9fe36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:42.295081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d4441-8663-4974-9097-9f493ab9fe36 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:42.303111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:42.329734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 833/1666] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:42:42.333603 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d4441-8663-4974-9097-9f493ab9fe36 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:42.355865 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d4441-8663-4974-9097-9f493ab9fe36 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:42.364634 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd9875-1cbe-44f4-9880-aeb6a9b3f4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:42.365552 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd9875-1cbe-44f4-9880-aeb6a9b3f4b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:42.367133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd9875-1cbe-44f4-9880-aeb6a9b3f4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:42.367493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd9875-1cbe-44f4-9880-aeb6a9b3f4b9 admin admin] PATH_INFO: `/v3/projects/be76f741eb4d43ff95ef7ca86d6cf594/users/e65f14431dc94fdf8768655d90eb9219/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:42.420747 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd9875-1cbe-44f4-9880-aeb6a9b3f4b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be76f741eb4d43ff95ef7ca86d6cf594, user_id=e65f14431dc94fdf8768655d90eb9219, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:42.437643 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd9875-1cbe-44f4-9880-aeb6a9b3f4b9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:42.485487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-977ada1a-8e87-43ee-b921-08fa1fcd3878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '808314251d5b4d38b40f4712effb03d7', 'request_id': 'req-977ada1a-8e87-43ee-b921-08fa1fcd3878'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:42.509163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-977ada1a-8e87-43ee-b921-08fa1fcd3878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '808314251d5b4d38b40f4712effb03d7', 'request_id': 'req-977ada1a-8e87-43ee-b921-08fa1fcd3878'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:42.556670 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-977ada1a-8e87-43ee-b921-08fa1fcd3878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '808314251d5b4d38b40f4712effb03d7', 'request_id': 'req-977ada1a-8e87-43ee-b921-08fa1fcd3878'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:42.568332 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 834/1667] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:42 2022] PUT /identity/v3/projects/be76f741eb4d43ff95ef7ca86d6cf594/users/e65f14431dc94fdf8768655d90eb9219/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:42.578774 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c9a62-fa0b-4d69-a86b-f1813126c2d9 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:42.579862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c9a62-fa0b-4d69-a86b-f1813126c2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:42.582389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c9a62-fa0b-4d69-a86b-f1813126c2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:42.673065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-977ada1a-8e87-43ee-b921-08fa1fcd3878 admin admin] The token cache is being invalidate because project 808314251d5b4d38b40f4712effb03d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:42:42.673345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-977ada1a-8e87-43ee-b921-08fa1fcd3878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:42.677913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-977ada1a-8e87-43ee-b921-08fa1fcd3878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:42.685531 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 834/1668] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:42:42 2022] DELETE /identity/v3/projects/808314251d5b4d38b40f4712effb03d7 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:42.690691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740e037-eb3b-4025-9af9-c4612ac65912 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:42.694088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740e037-eb3b-4025-9af9-c4612ac65912 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:42.694819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740e037-eb3b-4025-9af9-c4612ac65912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:42.770050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-b740e037-eb3b-4025-9af9-c4612ac65912 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:42.779151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-996c9a62-fa0b-4d69-a86b-f1813126c2d9 None None] MFA Rules not processed for user `e65f14431dc94fdf8768655d90eb9219`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:42.780513 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-996c9a62-fa0b-4d69-a86b-f1813126c2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:42.797031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b740e037-eb3b-4025-9af9-c4612ac65912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:43.090511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 835/1669] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:42:43.100883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf7232-f330-48a1-9996-8ccbef2f523a None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:43.101182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf7232-f330-48a1-9996-8ccbef2f523a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:43.101460 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cf7232-f330-48a1-9996-8ccbef2f523a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:43.102142 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:43.122912 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 835/1670] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 545 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:43.128769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c9a62-fa0b-4d69-a86b-f1813126c2d9 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:43.130947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-996c9a62-fa0b-4d69-a86b-f1813126c2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:43.161324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-b3cf7232-f330-48a1-9996-8ccbef2f523a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:43.162735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b3cf7232-f330-48a1-9996-8ccbef2f523a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:43.170934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c9a62-fa0b-4d69-a86b-f1813126c2d9 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:43.189127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bfcdc-0f5b-4728-8f68-e0da6efd7ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:43.189770 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bfcdc-0f5b-4728-8f68-e0da6efd7ede admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:43.190074 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bfcdc-0f5b-4728-8f68-e0da6efd7ede admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:43.190731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bfcdc-0f5b-4728-8f68-e0da6efd7ede admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:43.191453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bfcdc-0f5b-4728-8f68-e0da6efd7ede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:43.195255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556bfcdc-0f5b-4728-8f68-e0da6efd7ede admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:43.197940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 836/1671] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:42:43.201795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf7232-f330-48a1-9996-8ccbef2f523a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:43.219458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cf7232-f330-48a1-9996-8ccbef2f523a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:43.245608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a1d90-73ac-4b15-af7d-6bf42c2bce4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:43.245608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a1d90-73ac-4b15-af7d-6bf42c2bce4d admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:43.245608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a1d90-73ac-4b15-af7d-6bf42c2bce4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:43.245608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a1d90-73ac-4b15-af7d-6bf42c2bce4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:43.245885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a1d90-73ac-4b15-af7d-6bf42c2bce4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:42:43.245885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a1d90-73ac-4b15-af7d-6bf42c2bce4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:43.245885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a1d90-73ac-4b15-af7d-6bf42c2bce4d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:43.261512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 837/1672] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:42:43.267239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a1d90-73ac-4b15-af7d-6bf42c2bce4d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:43.291455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a1d90-73ac-4b15-af7d-6bf42c2bce4d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:43.306345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f7200-152e-4b67-9a69-64229021ec63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:43.308394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f7200-152e-4b67-9a69-64229021ec63 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:43.308804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f7200-152e-4b67-9a69-64229021ec63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:43.309194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f7200-152e-4b67-9a69-64229021ec63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:43.319300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f7200-152e-4b67-9a69-64229021ec63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:43.323271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 836/1673] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:43 2022] POST /identity/v3/projects => generated 379 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:42:43.330203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bfcdc-0f5b-4728-8f68-e0da6efd7ede admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:43.332308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f7200-152e-4b67-9a69-64229021ec63 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:43.456802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 838/1674] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:43 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:43.461671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f7200-152e-4b67-9a69-64229021ec63 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:43.472065 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bfcdc-0f5b-4728-8f68-e0da6efd7ede admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:43.599641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:43.602735 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79413b80-af7d-45a8-97e1-4b2592c93e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:43.603360 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79413b80-af7d-45a8-97e1-4b2592c93e99 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:43.603629 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79413b80-af7d-45a8-97e1-4b2592c93e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:43.603894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79413b80-af7d-45a8-97e1-4b2592c93e99 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:43.606163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79413b80-af7d-45a8-97e1-4b2592c93e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:43.611853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79413b80-af7d-45a8-97e1-4b2592c93e99 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:43.625155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f7200-152e-4b67-9a69-64229021ec63 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:43.644201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ecdfd-5acb-4059-892e-ef344d81675e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:43.646341 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ecdfd-5acb-4059-892e-ef344d81675e admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:43.646341 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ecdfd-5acb-4059-892e-ef344d81675e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:43.646341 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ecdfd-5acb-4059-892e-ef344d81675e admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:43.646341 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ecdfd-5acb-4059-892e-ef344d81675e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:43.647983 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ecdfd-5acb-4059-892e-ef344d81675e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:43.743439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 837/1675] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:43 2022] POST /identity/v3/users => generated 296 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:43.747216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79413b80-af7d-45a8-97e1-4b2592c93e99 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:43.770768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 839/1676] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:43 2022] POST /identity/v3/users => generated 290 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:42:43.780955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ecdfd-5acb-4059-892e-ef344d81675e admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:43.781795 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79413b80-af7d-45a8-97e1-4b2592c93e99 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:43.804551 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01257c-21e7-4384-896a-aa052d2aebd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:43.805595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ecdfd-5acb-4059-892e-ef344d81675e admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:43.806726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01257c-21e7-4384-896a-aa052d2aebd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:43.811950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01257c-21e7-4384-896a-aa052d2aebd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:43.811950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01257c-21e7-4384-896a-aa052d2aebd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:43.811950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01257c-21e7-4384-896a-aa052d2aebd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:43.811950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01257c-21e7-4384-896a-aa052d2aebd4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:43.826508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9c7e6-091a-4c96-84d6-9098cdddc7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:43.827562 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9c7e6-091a-4c96-84d6-9098cdddc7ba admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:43.827980 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9c7e6-091a-4c96-84d6-9098cdddc7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:43.828386 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9c7e6-091a-4c96-84d6-9098cdddc7ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:43.829178 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9c7e6-091a-4c96-84d6-9098cdddc7ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:43.833215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9c7e6-091a-4c96-84d6-9098cdddc7ba admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:43.843274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 838/1677] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:42:43.850844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01257c-21e7-4384-896a-aa052d2aebd4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:43.875150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 840/1678] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:43.876121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9c7e6-091a-4c96-84d6-9098cdddc7ba admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:43.884304 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01257c-21e7-4384-896a-aa052d2aebd4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:43.903627 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75764ada-c9c4-433b-9747-e910b4dac16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:43.904332 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75764ada-c9c4-433b-9747-e910b4dac16e admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:43.904571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75764ada-c9c4-433b-9747-e910b4dac16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:43.904861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75764ada-c9c4-433b-9747-e910b4dac16e admin admin] PATH_INFO: `/v3/projects/d25a4129bfd24e0e9128c1e7f6a2b6cd/users/fba0eb27c84640709d417d60804aa951/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:43.906981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9c7e6-091a-4c96-84d6-9098cdddc7ba admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:43.948028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6b7a2-2e6a-46be-998f-0c356d5d8e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:43.949329 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6b7a2-2e6a-46be-998f-0c356d5d8e1f admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:43.949861 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6b7a2-2e6a-46be-998f-0c356d5d8e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:43.950351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6b7a2-2e6a-46be-998f-0c356d5d8e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:43.951211 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6b7a2-2e6a-46be-998f-0c356d5d8e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:43.953080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6b7a2-2e6a-46be-998f-0c356d5d8e1f admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:43.960149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75764ada-c9c4-433b-9747-e910b4dac16e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d25a4129bfd24e0e9128c1e7f6a2b6cd, user_id=fba0eb27c84640709d417d60804aa951, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:43.963310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75764ada-c9c4-433b-9747-e910b4dac16e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:43.998637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 841/1679] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:44.005224 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6b7a2-2e6a-46be-998f-0c356d5d8e1f admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:44.037743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6b7a2-2e6a-46be-998f-0c356d5d8e1f admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:44.046684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 839/1680] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:43 2022] PUT /identity/v3/projects/d25a4129bfd24e0e9128c1e7f6a2b6cd/users/fba0eb27c84640709d417d60804aa951/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:44.052014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75764ada-c9c4-433b-9747-e910b4dac16e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:44.140829 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:44.235371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75764ada-c9c4-433b-9747-e910b4dac16e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:44.246558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9b8c0-0d34-41db-ab50-4b1fb5e9371e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:44.247190 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9b8c0-0d34-41db-ab50-4b1fb5e9371e admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:44.247463 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9b8c0-0d34-41db-ab50-4b1fb5e9371e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:44.247731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9b8c0-0d34-41db-ab50-4b1fb5e9371e admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:44.249033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9b8c0-0d34-41db-ab50-4b1fb5e9371e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:44.250985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc552776-84a5-448a-88fb-1783de78d114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:44.251755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc552776-84a5-448a-88fb-1783de78d114 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:44.252024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc552776-84a5-448a-88fb-1783de78d114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:44.252286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc552776-84a5-448a-88fb-1783de78d114 admin admin] PATH_INFO: `/v3/projects/743739cecc414beba6faaa16a378a9a0/users/499c76edefeb47589181acceed0db46c/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:44.257454 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9b8c0-0d34-41db-ab50-4b1fb5e9371e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:44.282130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 840/1681] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:42:44.286200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9b8c0-0d34-41db-ab50-4b1fb5e9371e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:44.291438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc552776-84a5-448a-88fb-1783de78d114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=743739cecc414beba6faaa16a378a9a0, user_id=499c76edefeb47589181acceed0db46c, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:44.299169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc552776-84a5-448a-88fb-1783de78d114 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:44.313624 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9b8c0-0d34-41db-ab50-4b1fb5e9371e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:44.320910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c58ee9-d49c-4980-ad34-4dddb6b3e892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:44.321868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c58ee9-d49c-4980-ad34-4dddb6b3e892 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:44.322325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c58ee9-d49c-4980-ad34-4dddb6b3e892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:44.324960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c58ee9-d49c-4980-ad34-4dddb6b3e892 admin admin] PATH_INFO: `/v3/projects/d25a4129bfd24e0e9128c1e7f6a2b6cd/users/fba0eb27c84640709d417d60804aa951/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:44.325744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c58ee9-d49c-4980-ad34-4dddb6b3e892 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d25a4129bfd24e0e9128c1e7f6a2b6cd, user_id=fba0eb27c84640709d417d60804aa951, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:44.330979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c58ee9-d49c-4980-ad34-4dddb6b3e892 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:44.344892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 841/1682] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:44 2022] PUT /identity/v3/projects/d25a4129bfd24e0e9128c1e7f6a2b6cd/users/fba0eb27c84640709d417d60804aa951/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:44.355831 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfccc0-93fc-4094-a2b4-fbbeb77902fd None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:44.356315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfccc0-93fc-4094-a2b4-fbbeb77902fd None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:44.356715 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfccc0-93fc-4094-a2b4-fbbeb77902fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:44.404544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 842/1683] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:44 2022] PUT /identity/v3/projects/743739cecc414beba6faaa16a378a9a0/users/499c76edefeb47589181acceed0db46c/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:44.406587 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fc518-567e-46d9-849c-ffb9c8a7f199 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:44.406988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fc518-567e-46d9-849c-ffb9c8a7f199 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:44.407373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fc518-567e-46d9-849c-ffb9c8a7f199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:44.429051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-b2bfccc0-93fc-4094-a2b4-fbbeb77902fd None None] MFA Rules not processed for user `fba0eb27c84640709d417d60804aa951`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:44.430475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b2bfccc0-93fc-4094-a2b4-fbbeb77902fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:44.471400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-6e2fc518-567e-46d9-849c-ffb9c8a7f199 None None] MFA Rules not processed for user `499c76edefeb47589181acceed0db46c`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:44.473398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6e2fc518-567e-46d9-849c-ffb9c8a7f199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:44.547190 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 842/1684] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:44 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:42:44.590397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfccc0-93fc-4094-a2b4-fbbeb77902fd None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:44.590397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b2bfccc0-93fc-4094-a2b4-fbbeb77902fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:44.661510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 843/1685] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:44.663350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441aef41-7a94-48da-80bd-a6b6d88b69af None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:44.663978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441aef41-7a94-48da-80bd-a6b6d88b69af None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:44.666236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441aef41-7a94-48da-80bd-a6b6d88b69af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:44.714127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfccc0-93fc-4094-a2b4-fbbeb77902fd None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:44.720549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-441aef41-7a94-48da-80bd-a6b6d88b69af None None] MFA Rules not processed for user `12e7e80742cc426da9984af75016a890`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:44.721341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347afdab-4311-4fb0-965f-7e3417814213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:44.722109 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347afdab-4311-4fb0-965f-7e3417814213 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:44.722491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347afdab-4311-4fb0-965f-7e3417814213 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:44.723426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347afdab-4311-4fb0-965f-7e3417814213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:44.725706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-441aef41-7a94-48da-80bd-a6b6d88b69af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:44.747680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347afdab-4311-4fb0-965f-7e3417814213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:44.751151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347afdab-4311-4fb0-965f-7e3417814213 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:44.783028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 843/1686] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:44 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:44.784866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347afdab-4311-4fb0-965f-7e3417814213 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:44.868645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 844/1687] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:44.880016 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb9d07-8215-4ea4-9a5c-4d1114fbb0a0 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:44.880455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb9d07-8215-4ea4-9a5c-4d1114fbb0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:44.881003 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb9d07-8215-4ea4-9a5c-4d1114fbb0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:44.935852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-56fb9d07-8215-4ea4-9a5c-4d1114fbb0a0 None None] MFA Rules not processed for user `12e7e80742cc426da9984af75016a890`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:44.936010 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347afdab-4311-4fb0-965f-7e3417814213 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:44.939446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-56fb9d07-8215-4ea4-9a5c-4d1114fbb0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:44.944029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684340d-e4eb-4c59-9c5e-87a2220a66d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:44.944932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684340d-e4eb-4c59-9c5e-87a2220a66d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:44.947086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684340d-e4eb-4c59-9c5e-87a2220a66d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:44.947504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684340d-e4eb-4c59-9c5e-87a2220a66d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:44.948563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684340d-e4eb-4c59-9c5e-87a2220a66d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:44.956652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684340d-e4eb-4c59-9c5e-87a2220a66d0 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:44.957667 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 845/1688] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:42:44.958772 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb9d07-8215-4ea4-9a5c-4d1114fbb0a0 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:44.960169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-56fb9d07-8215-4ea4-9a5c-4d1114fbb0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:44.989385 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb9d07-8215-4ea4-9a5c-4d1114fbb0a0 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:44.999759 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6faff4-f6bd-4834-93f6-e9aa193135e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:45.000444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6faff4-f6bd-4834-93f6-e9aa193135e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:45.000724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6faff4-f6bd-4834-93f6-e9aa193135e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:45.000990 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6faff4-f6bd-4834-93f6-e9aa193135e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:45.005580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-cf6faff4-f6bd-4834-93f6-e9aa193135e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:45.090916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 844/1689] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:44 2022] POST /identity/v3/projects => generated 379 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:45.092187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684340d-e4eb-4c59-9c5e-87a2220a66d0 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:45.098833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8684340d-e4eb-4c59-9c5e-87a2220a66d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:45.166061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6faff4-f6bd-4834-93f6-e9aa193135e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:45.167531 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6faff4-f6bd-4834-93f6-e9aa193135e4 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:45.197123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 846/1690] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:45.197950 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6faff4-f6bd-4834-93f6-e9aa193135e4 service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:45.220649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6faff4-f6bd-4834-93f6-e9aa193135e4 service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:45.313649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684340d-e4eb-4c59-9c5e-87a2220a66d0 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:45.315062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba1cfa-a3e7-45d7-a0c0-ebef9820e209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:45.315955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba1cfa-a3e7-45d7-a0c0-ebef9820e209 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:45.316598 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba1cfa-a3e7-45d7-a0c0-ebef9820e209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:45.323227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba1cfa-a3e7-45d7-a0c0-ebef9820e209 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:45.324929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba1cfa-a3e7-45d7-a0c0-ebef9820e209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:45.327745 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba1cfa-a3e7-45d7-a0c0-ebef9820e209 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:45.331660 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711170c6-19b2-4619-beef-9e7944d5a9cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:45.336602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711170c6-19b2-4619-beef-9e7944d5a9cb service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:45.336602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711170c6-19b2-4619-beef-9e7944d5a9cb service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:45.336602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711170c6-19b2-4619-beef-9e7944d5a9cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:45.431655 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 847/1691] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:45 2022] POST /identity/v3/users => generated 296 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:45.440657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba1cfa-a3e7-45d7-a0c0-ebef9820e209 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:45.465450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba1cfa-a3e7-45d7-a0c0-ebef9820e209 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:45.472730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711170c6-19b2-4619-beef-9e7944d5a9cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:45.474575 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711170c6-19b2-4619-beef-9e7944d5a9cb service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:45.484740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42373d2d-7b0d-4a6a-a099-20be22519565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:45.485792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42373d2d-7b0d-4a6a-a099-20be22519565 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:45.486321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42373d2d-7b0d-4a6a-a099-20be22519565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:45.487608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42373d2d-7b0d-4a6a-a099-20be22519565 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:45.496735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42373d2d-7b0d-4a6a-a099-20be22519565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:45.499274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 845/1692] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:45.500641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42373d2d-7b0d-4a6a-a099-20be22519565 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:45.503636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:45.524459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 848/1693] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:45.533497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711170c6-19b2-4619-beef-9e7944d5a9cb service nova] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:45.542000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:45.556733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711170c6-19b2-4619-beef-9e7944d5a9cb service nova] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:45.575323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b098b3c-0871-4736-8d8c-db333714bbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:45.575524 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b098b3c-0871-4736-8d8c-db333714bbc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:45.575798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b098b3c-0871-4736-8d8c-db333714bbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:45.576067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b098b3c-0871-4736-8d8c-db333714bbc7 admin admin] PATH_INFO: `/v3/projects/d18b47746c9147a7aee6cbd3dd70fe45/users/eec0f48003ae4674948012c8d541d282/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:45.649030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b098b3c-0871-4736-8d8c-db333714bbc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d18b47746c9147a7aee6cbd3dd70fe45, user_id=eec0f48003ae4674948012c8d541d282, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:45.655526 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b098b3c-0871-4736-8d8c-db333714bbc7 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:45.788803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 846/1694] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:45 2022] PUT /identity/v3/projects/d18b47746c9147a7aee6cbd3dd70fe45/users/eec0f48003ae4674948012c8d541d282/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:45.791545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42373d2d-7b0d-4a6a-a099-20be22519565 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:45.814025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42373d2d-7b0d-4a6a-a099-20be22519565 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:45.866793 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b098b3c-0871-4736-8d8c-db333714bbc7 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:45.870264 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7b098b3c-0871-4736-8d8c-db333714bbc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:46.064537 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84b934-20ef-4a97-ac12-54f2d06522f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:46.066197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84b934-20ef-4a97-ac12-54f2d06522f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:46.069714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84b934-20ef-4a97-ac12-54f2d06522f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:46.071443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84b934-20ef-4a97-ac12-54f2d06522f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:46.073034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84b934-20ef-4a97-ac12-54f2d06522f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:46.075687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84b934-20ef-4a97-ac12-54f2d06522f5 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:46.104239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 849/1695] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:46.105632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b098b3c-0871-4736-8d8c-db333714bbc7 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:46.106678 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84b934-20ef-4a97-ac12-54f2d06522f5 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:46.128806 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84b934-20ef-4a97-ac12-54f2d06522f5 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:46.130939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:46.131603 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:46.132767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:46.133206 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] PATH_INFO: `/v3/users/050a6123ac274eb1a6cf35560cc9d1c0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:46.134674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=050a6123ac274eb1a6cf35560cc9d1c0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:46.136867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:46.143368 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a9e99-8b98-4824-8d42-e4c9aea1cc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:46.147855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a9e99-8b98-4824-8d42-e4c9aea1cc09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:46.148204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a9e99-8b98-4824-8d42-e4c9aea1cc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:46.148591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a9e99-8b98-4824-8d42-e4c9aea1cc09 admin admin] PATH_INFO: `/v3/projects/d18b47746c9147a7aee6cbd3dd70fe45/users/eec0f48003ae4674948012c8d541d282/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:46.155558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a9e99-8b98-4824-8d42-e4c9aea1cc09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d18b47746c9147a7aee6cbd3dd70fe45, user_id=eec0f48003ae4674948012c8d541d282, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:46.161040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a9e99-8b98-4824-8d42-e4c9aea1cc09 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:46.175353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 850/1696] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:46 2022] PUT /identity/v3/projects/d18b47746c9147a7aee6cbd3dd70fe45/users/eec0f48003ae4674948012c8d541d282/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:46.194020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c4d64-8bb2-4633-bb09-8b0b5701f85e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:46.194859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c4d64-8bb2-4633-bb09-8b0b5701f85e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:46.194859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c4d64-8bb2-4633-bb09-8b0b5701f85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:46.250237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-ea9c4d64-8bb2-4633-bb09-8b0b5701f85e None None] MFA Rules not processed for user `eec0f48003ae4674948012c8d541d282`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:46.255026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-ea9c4d64-8bb2-4633-bb09-8b0b5701f85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:46.450965 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '050a6123ac274eb1a6cf35560cc9d1c0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:46.456651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 851/1697] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:46.458046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '050a6123ac274eb1a6cf35560cc9d1c0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:46.463281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c4d64-8bb2-4633-bb09-8b0b5701f85e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:46.568684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c4d64-8bb2-4633-bb09-8b0b5701f85e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:46.573193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb8b57e-c726-4b86-829a-ee248f8dfc42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:46.574150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8b57e-c726-4b86-829a-ee248f8dfc42 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:46.574638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8b57e-c726-4b86-829a-ee248f8dfc42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:46.575136 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb8b57e-c726-4b86-829a-ee248f8dfc42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:46.587958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb8b57e-c726-4b86-829a-ee248f8dfc42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:46.590584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb8b57e-c726-4b86-829a-ee248f8dfc42 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:46.608791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 852/1698] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:46.650832 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '050a6123ac274eb1a6cf35560cc9d1c0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:46.651688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '050a6123ac274eb1a6cf35560cc9d1c0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:46.661624 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '050a6123ac274eb1a6cf35560cc9d1c0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:46.735973 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:46.737277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 847/1699] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:42:45 2022] DELETE /identity/v3/users/050a6123ac274eb1a6cf35560cc9d1c0 => generated 0 bytes in 873 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:46.743710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd2fca-c476-4fc4-b904-dd5ae8e2467c None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:46.744214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd2fca-c476-4fc4-b904-dd5ae8e2467c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:46.744668 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd2fca-c476-4fc4-b904-dd5ae8e2467c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:46.791916 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-a8dd2fca-c476-4fc4-b904-dd5ae8e2467c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:46.793444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:46.794289 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a8dd2fca-c476-4fc4-b904-dd5ae8e2467c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:46.898051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:46.899759 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:47.034932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 853/1700] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:42:47.112444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a259b-f4b6-4d47-96a6-9755ce23d4e8 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:47.121540 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29ca3a-e43a-4125-98b6-1615409adc72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:47.124717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29ca3a-e43a-4125-98b6-1615409adc72 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:47.125242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29ca3a-e43a-4125-98b6-1615409adc72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:47.125626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29ca3a-e43a-4125-98b6-1615409adc72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:47.135102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-db29ca3a-e43a-4125-98b6-1615409adc72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:47.159943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd2fca-c476-4fc4-b904-dd5ae8e2467c None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:47.162585 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a8dd2fca-c476-4fc4-b904-dd5ae8e2467c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:47.352365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29ca3a-e43a-4125-98b6-1615409adc72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:47.357413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd2fca-c476-4fc4-b904-dd5ae8e2467c None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:47.359020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29ca3a-e43a-4125-98b6-1615409adc72 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:47.374684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638949b-9305-448d-84f8-f8b9418f8f8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:47.378850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638949b-9305-448d-84f8-f8b9418f8f8c service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:47.382224 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638949b-9305-448d-84f8-f8b9418f8f8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:47.382553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638949b-9305-448d-84f8-f8b9418f8f8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:47.391133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 848/1701] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:42:47.411730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638949b-9305-448d-84f8-f8b9418f8f8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:47.413393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638949b-9305-448d-84f8-f8b9418f8f8c service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:47.454683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 854/1702] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:42:47.593044 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df36140-1ee2-4493-b675-37bd00e29572 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:47.593593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df36140-1ee2-4493-b675-37bd00e29572 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:47.593816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df36140-1ee2-4493-b675-37bd00e29572 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:47.595762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 849/1703] 10.222.0.49 () {54 vars in 912 bytes} [Fri Sep 16 05:42:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:42:47.601629 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b4d69-1f4b-4650-8f30-54a01aab6c03 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:47.602209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b4d69-1f4b-4650-8f30-54a01aab6c03 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:47.602629 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b4d69-1f4b-4650-8f30-54a01aab6c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:47.677263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-826b4d69-1f4b-4650-8f30-54a01aab6c03 None None] MFA Rules not processed for user `3e923be9aeed43728690d24d85dbe062`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:47.680911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-826b4d69-1f4b-4650-8f30-54a01aab6c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:47.702219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 855/1704] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:42:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:47.742365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df36140-1ee2-4493-b675-37bd00e29572 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:47.744267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2df36140-1ee2-4493-b675-37bd00e29572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:47.766855 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df36140-1ee2-4493-b675-37bd00e29572 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:47.775550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:47.776592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a30bf8-7163-4da4-9fe6-e939252f0b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:47.777580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a30bf8-7163-4da4-9fe6-e939252f0b3e service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:47.778101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a30bf8-7163-4da4-9fe6-e939252f0b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:47.778507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a30bf8-7163-4da4-9fe6-e939252f0b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:47.794655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a30bf8-7163-4da4-9fe6-e939252f0b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:47.795825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a30bf8-7163-4da4-9fe6-e939252f0b3e service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:47.812921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 850/1705] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:48.042852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcfde8-cc3d-489a-83cb-983f60098202 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:48.043305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcfde8-cc3d-489a-83cb-983f60098202 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:48.043723 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcfde8-cc3d-489a-83cb-983f60098202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:48.113047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-86fcfde8-cc3d-489a-83cb-983f60098202 None None] MFA Rules not processed for user `edd619bf865645fab0460d3aa847a734`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:48.117247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-86fcfde8-cc3d-489a-83cb-983f60098202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:48.346293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 856/1706] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:48.361859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a30bf8-7163-4da4-9fe6-e939252f0b3e service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:48.365000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-44a30bf8-7163-4da4-9fe6-e939252f0b3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:48.371587 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:48.455651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fcfde8-cc3d-489a-83cb-983f60098202 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:48.457701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-86fcfde8-cc3d-489a-83cb-983f60098202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:48.501221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a30bf8-7163-4da4-9fe6-e939252f0b3e service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:48.520699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4339cd0-5c12-419b-a836-9121b0bf92ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:48.522058 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4339cd0-5c12-419b-a836-9121b0bf92ff service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:48.522449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4339cd0-5c12-419b-a836-9121b0bf92ff service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:48.522744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4339cd0-5c12-419b-a836-9121b0bf92ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:48.531563 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fcfde8-cc3d-489a-83cb-983f60098202 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:48.533726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4339cd0-5c12-419b-a836-9121b0bf92ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:48.537006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4339cd0-5c12-419b-a836-9121b0bf92ff service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:48.553170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 851/1707] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:42:48.565265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bba137-bcbf-46fa-823f-94c00893716f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:48.571538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bba137-bcbf-46fa-823f-94c00893716f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:48.576139 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bba137-bcbf-46fa-823f-94c00893716f admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:48.576491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bba137-bcbf-46fa-823f-94c00893716f admin admin] PATH_INFO: `/v3/projects/ce6e4d8568944cc78a53f87dec556ec1` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:48.579046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bba137-bcbf-46fa-823f-94c00893716f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce6e4d8568944cc78a53f87dec556ec1)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:48.589184 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bba137-bcbf-46fa-823f-94c00893716f admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:48.823038 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-74bba137-bcbf-46fa-823f-94c00893716f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce6e4d8568944cc78a53f87dec556ec1', 'request_id': 'req-74bba137-bcbf-46fa-823f-94c00893716f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:48.846291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-74bba137-bcbf-46fa-823f-94c00893716f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce6e4d8568944cc78a53f87dec556ec1', 'request_id': 'req-74bba137-bcbf-46fa-823f-94c00893716f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:48.860852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-74bba137-bcbf-46fa-823f-94c00893716f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce6e4d8568944cc78a53f87dec556ec1', 'request_id': 'req-74bba137-bcbf-46fa-823f-94c00893716f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:48.975995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-74bba137-bcbf-46fa-823f-94c00893716f admin admin] The token cache is being invalidate because project ce6e4d8568944cc78a53f87dec556ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:42:48.976203 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-74bba137-bcbf-46fa-823f-94c00893716f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:48.977146 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-74bba137-bcbf-46fa-823f-94c00893716f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:48.979257 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 857/1708] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:42:48 2022] DELETE /identity/v3/projects/ce6e4d8568944cc78a53f87dec556ec1 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:48.983943 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688911d-864f-4b8d-ae3f-9d2a906829cb None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:48.984445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688911d-864f-4b8d-ae3f-9d2a906829cb None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:48.984519 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688911d-864f-4b8d-ae3f-9d2a906829cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:49.036268 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-0688911d-864f-4b8d-ae3f-9d2a906829cb None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:49.037965 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0688911d-864f-4b8d-ae3f-9d2a906829cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:49.188861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 852/1709] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:49.198862 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a3c6e-014d-42ab-8eed-55b4bb85b002 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:49.199196 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a3c6e-014d-42ab-8eed-55b4bb85b002 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:49.199506 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a3c6e-014d-42ab-8eed-55b4bb85b002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:49.278509 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-bb2a3c6e-014d-42ab-8eed-55b4bb85b002 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:49.280813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-bb2a3c6e-014d-42ab-8eed-55b4bb85b002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:49.291296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 858/1710] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:49.295481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688911d-864f-4b8d-ae3f-9d2a906829cb None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:49.334837 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688911d-864f-4b8d-ae3f-9d2a906829cb None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:49.348072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:49.349626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080634d-6372-4b26-9c10-cc2e2acff4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:49.350339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080634d-6372-4b26-9c10-cc2e2acff4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:49.350580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080634d-6372-4b26-9c10-cc2e2acff4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:49.351011 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1080634d-6372-4b26-9c10-cc2e2acff4b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:49.351871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080634d-6372-4b26-9c10-cc2e2acff4b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:42:49.352333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080634d-6372-4b26-9c10-cc2e2acff4b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:49.360273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1080634d-6372-4b26-9c10-cc2e2acff4b0 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:49.416818 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 853/1711] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:42:49.425138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a3c6e-014d-42ab-8eed-55b4bb85b002 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:49.458244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a3c6e-014d-42ab-8eed-55b4bb85b002 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:49.509686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9186f7-d551-4a2e-8c39-bb116c43eb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:49.510427 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9186f7-d551-4a2e-8c39-bb116c43eb31 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:49.510806 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9186f7-d551-4a2e-8c39-bb116c43eb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:49.511169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9186f7-d551-4a2e-8c39-bb116c43eb31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:49.512298 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9186f7-d551-4a2e-8c39-bb116c43eb31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:49.525620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9186f7-d551-4a2e-8c39-bb116c43eb31 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:49.684297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 859/1712] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:49 2022] POST /identity/v3/projects => generated 385 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:49.684980 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080634d-6372-4b26-9c10-cc2e2acff4b0 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:49.711231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1080634d-6372-4b26-9c10-cc2e2acff4b0 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:49.938843 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb43e60-e290-4541-8844-bab78a622527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:49.940019 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb43e60-e290-4541-8844-bab78a622527 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:49.940432 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb43e60-e290-4541-8844-bab78a622527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:49.940898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb43e60-e290-4541-8844-bab78a622527 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:49.942167 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb43e60-e290-4541-8844-bab78a622527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:49.945943 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb43e60-e290-4541-8844-bab78a622527 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:50.084692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 854/1713] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:49 2022] POST /identity/v3/users => generated 293 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:50.089148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9186f7-d551-4a2e-8c39-bb116c43eb31 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:50.115064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9186f7-d551-4a2e-8c39-bb116c43eb31 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:50.151738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f513ad-4cae-48c3-8d3b-1688a39bdea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:50.155553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f513ad-4cae-48c3-8d3b-1688a39bdea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:50.158968 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f513ad-4cae-48c3-8d3b-1688a39bdea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:50.159501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f513ad-4cae-48c3-8d3b-1688a39bdea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:50.161281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f513ad-4cae-48c3-8d3b-1688a39bdea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:50.169499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f513ad-4cae-48c3-8d3b-1688a39bdea3 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:50.203839 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb43e60-e290-4541-8844-bab78a622527 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:50.205664 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0fb43e60-e290-4541-8844-bab78a622527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:50.237618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 860/1714] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:50.241661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f513ad-4cae-48c3-8d3b-1688a39bdea3 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:50.258399 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f513ad-4cae-48c3-8d3b-1688a39bdea3 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:50.268177 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc261262-e748-43b0-b4d8-2240ecd96d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:50.268797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc261262-e748-43b0-b4d8-2240ecd96d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:50.269068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc261262-e748-43b0-b4d8-2240ecd96d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:50.269438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc261262-e748-43b0-b4d8-2240ecd96d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:50.270307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc261262-e748-43b0-b4d8-2240ecd96d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:50.271907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc261262-e748-43b0-b4d8-2240ecd96d0b admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:50.297967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 861/1715] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:50.301863 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc261262-e748-43b0-b4d8-2240ecd96d0b admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:50.325039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc261262-e748-43b0-b4d8-2240ecd96d0b admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:50.344925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020aa0ab-8454-461a-a146-47c3052b268a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:50.346088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020aa0ab-8454-461a-a146-47c3052b268a admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:50.346541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020aa0ab-8454-461a-a146-47c3052b268a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:50.347128 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020aa0ab-8454-461a-a146-47c3052b268a admin admin] PATH_INFO: `/v3/projects/936662ef12c046d98873d6efccaa17ab/users/4431008e78b34bc0bfc515d4125e8ae1/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:50.395484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020aa0ab-8454-461a-a146-47c3052b268a admin admin] RBAC: Authorizing `identity:create_grant(project_id=936662ef12c046d98873d6efccaa17ab, user_id=4431008e78b34bc0bfc515d4125e8ae1, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:50.402801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020aa0ab-8454-461a-a146-47c3052b268a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:50.424791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb43e60-e290-4541-8844-bab78a622527 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:50.438560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8511961-f2d7-48e4-98df-5979e1305be2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9975c0dab76f4ccbb4d2107e8e780ae5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:50.439192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8511961-f2d7-48e4-98df-5979e1305be2 service placement] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:50.439472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8511961-f2d7-48e4-98df-5979e1305be2 service placement] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:50.439742 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8511961-f2d7-48e4-98df-5979e1305be2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:50.445045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b8511961-f2d7-48e4-98df-5979e1305be2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:50.474843 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8511961-f2d7-48e4-98df-5979e1305be2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:50.482682 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8511961-f2d7-48e4-98df-5979e1305be2 service placement] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:50.493526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 862/1716] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:50 2022] PUT /identity/v3/projects/936662ef12c046d98873d6efccaa17ab/users/4431008e78b34bc0bfc515d4125e8ae1/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:50.495354 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883bc7d-cc07-4622-92dc-b40aae424ae6 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:50.495871 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883bc7d-cc07-4622-92dc-b40aae424ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:50.496282 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2883bc7d-cc07-4622-92dc-b40aae424ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:50.502358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 855/1717] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:42:50.506196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664d101-0d60-455a-84d2-9529396fc3dc None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:50.506739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664d101-0d60-455a-84d2-9529396fc3dc None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:50.510485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2664d101-0d60-455a-84d2-9529396fc3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:50.549865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-2883bc7d-cc07-4622-92dc-b40aae424ae6 None None] MFA Rules not processed for user `12e7e80742cc426da9984af75016a890`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:50.553924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2883bc7d-cc07-4622-92dc-b40aae424ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:50.572671 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-2664d101-0d60-455a-84d2-9529396fc3dc None None] MFA Rules not processed for user `c205ee9f85924da79b870648698caf64`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:50.574003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2664d101-0d60-455a-84d2-9529396fc3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:50.719669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 863/1718] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:50.738891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e0b58-a209-4335-8f70-6c9132e519e1 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:50.739427 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e0b58-a209-4335-8f70-6c9132e519e1 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:50.741557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e0b58-a209-4335-8f70-6c9132e519e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:50.800580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 856/1719] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:50.813284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664d101-0d60-455a-84d2-9529396fc3dc None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:50.819539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2664d101-0d60-455a-84d2-9529396fc3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:50.848797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-fb0e0b58-a209-4335-8f70-6c9132e519e1 None None] MFA Rules not processed for user `4431008e78b34bc0bfc515d4125e8ae1`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:50.850328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-fb0e0b58-a209-4335-8f70-6c9132e519e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:51.096706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2664d101-0d60-455a-84d2-9529396fc3dc None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:51.108041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:51.116231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043d8ed-e531-4dba-a0c3-3fe93ebf6bf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:51.117531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043d8ed-e531-4dba-a0c3-3fe93ebf6bf0 service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:51.117531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043d8ed-e531-4dba-a0c3-3fe93ebf6bf0 service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:51.117531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043d8ed-e531-4dba-a0c3-3fe93ebf6bf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:51.143466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043d8ed-e531-4dba-a0c3-3fe93ebf6bf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:51.153926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b043d8ed-e531-4dba-a0c3-3fe93ebf6bf0 service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:51.155122 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 864/1720] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:51.157593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3d0e7-990b-4577-b90b-d042cc62bed8 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:51.157905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3d0e7-990b-4577-b90b-d042cc62bed8 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:51.158176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3d0e7-990b-4577-b90b-d042cc62bed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:51.182176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 857/1721] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:51.184092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043d8ed-e531-4dba-a0c3-3fe93ebf6bf0 service nova] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:51.292000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:51.302020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c9f3d0e7-990b-4577-b90b-d042cc62bed8 None None] MFA Rules not processed for user `8e28d24465df4f319bc07406586789fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:51.303252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c9f3d0e7-990b-4577-b90b-d042cc62bed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:51.379849 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043d8ed-e531-4dba-a0c3-3fe93ebf6bf0 service nova] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:51.401083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088675a-067a-4917-8fb0-fabf0c07f225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:51.401943 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088675a-067a-4917-8fb0-fabf0c07f225 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:51.402388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088675a-067a-4917-8fb0-fabf0c07f225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:51.402799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088675a-067a-4917-8fb0-fabf0c07f225 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:51.403853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088675a-067a-4917-8fb0-fabf0c07f225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:51.406639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088675a-067a-4917-8fb0-fabf0c07f225 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:51.518509 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 865/1722] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:51.520773 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3d0e7-990b-4577-b90b-d042cc62bed8 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:51.539409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c9f3d0e7-990b-4577-b90b-d042cc62bed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:51.581001 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3d0e7-990b-4577-b90b-d042cc62bed8 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:51.586868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:51.587672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:51.588062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:51.588460 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] PATH_INFO: `/v3/users/c945ac8de66e4807bd82e0a63f20e3d5` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:51.589960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c945ac8de66e4807bd82e0a63f20e3d5)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:51.592590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:51.644420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 858/1723] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:51 2022] POST /identity/v3/projects => generated 385 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:51.646558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63be87-1913-4174-a868-d91c3c51687f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:51.646997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63be87-1913-4174-a868-d91c3c51687f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:51.647339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac63be87-1913-4174-a868-d91c3c51687f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:51.681309 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-ac63be87-1913-4174-a868-d91c3c51687f None None] MFA Rules not processed for user `8e28d24465df4f319bc07406586789fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:51.682609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ac63be87-1913-4174-a868-d91c3c51687f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:51.834606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 859/1724] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:51.845912 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63be87-1913-4174-a868-d91c3c51687f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:51.873874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c945ac8de66e4807bd82e0a63f20e3d5'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:51.874830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c945ac8de66e4807bd82e0a63f20e3d5'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:51.987743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c945ac8de66e4807bd82e0a63f20e3d5'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:52.004156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c945ac8de66e4807bd82e0a63f20e3d5'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:52.005432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c945ac8de66e4807bd82e0a63f20e3d5'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:52.012859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac63be87-1913-4174-a868-d91c3c51687f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:52.027181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22122849-0bb3-420f-8dea-3bad1f39fa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:52.030013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22122849-0bb3-420f-8dea-3bad1f39fa95 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:52.030568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22122849-0bb3-420f-8dea-3bad1f39fa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:52.031690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22122849-0bb3-420f-8dea-3bad1f39fa95 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:52.033287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22122849-0bb3-420f-8dea-3bad1f39fa95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:52.036464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22122849-0bb3-420f-8dea-3bad1f39fa95 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:52.038071 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 866/1725] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:42:51 2022] DELETE /identity/v3/users/c945ac8de66e4807bd82e0a63f20e3d5 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:52.039608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:52.042133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:52.205528 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 860/1726] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:42:51 2022] POST /identity/v3/users => generated 299 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:42:52.209942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524e2b3-e3b9-46cc-a976-1b04c12ada67 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:52.212498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524e2b3-e3b9-46cc-a976-1b04c12ada67 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:52.212933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524e2b3-e3b9-46cc-a976-1b04c12ada67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:52.222175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5da3f-2e30-4377-b9a9-a4ea7fd90648 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:52.234453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22912268-2bf9-4d6b-9279-0d2a7e2a0924 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:52.235264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22912268-2bf9-4d6b-9279-0d2a7e2a0924 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:52.235729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22912268-2bf9-4d6b-9279-0d2a7e2a0924 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:52.236075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22912268-2bf9-4d6b-9279-0d2a7e2a0924 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:52.238688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-22912268-2bf9-4d6b-9279-0d2a7e2a0924 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:52.268542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-6524e2b3-e3b9-46cc-a976-1b04c12ada67 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:52.270436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-6524e2b3-e3b9-46cc-a976-1b04c12ada67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:52.443868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22912268-2bf9-4d6b-9279-0d2a7e2a0924 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:52.450072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22912268-2bf9-4d6b-9279-0d2a7e2a0924 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:52.503144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 867/1727] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:52.506228 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22912268-2bf9-4d6b-9279-0d2a7e2a0924 service nova] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:52.507964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-22912268-2bf9-4d6b-9279-0d2a7e2a0924 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:52.529005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22912268-2bf9-4d6b-9279-0d2a7e2a0924 service nova] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:52.537124 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4939c56-4860-44ec-a3fc-6b77c4932975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:52.538138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4939c56-4860-44ec-a3fc-6b77c4932975 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:52.538636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4939c56-4860-44ec-a3fc-6b77c4932975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:52.538861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 861/1728] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:52.543854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4939c56-4860-44ec-a3fc-6b77c4932975 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:52.544574 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4939c56-4860-44ec-a3fc-6b77c4932975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:52.549445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4939c56-4860-44ec-a3fc-6b77c4932975 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:52.581682 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524e2b3-e3b9-46cc-a976-1b04c12ada67 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:52.590835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 868/1729] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:42:52.595129 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524e2b3-e3b9-46cc-a976-1b04c12ada67 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:52.614710 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565332fe-5fff-4cd9-b710-f91689271f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:52.615406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565332fe-5fff-4cd9-b710-f91689271f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:52.615684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565332fe-5fff-4cd9-b710-f91689271f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:52.615953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565332fe-5fff-4cd9-b710-f91689271f14 admin admin] PATH_INFO: `/v3/projects/21198d80eccc48d4bdc9f4a625eb1e11/users/97fa9ff620684c1b8a56044d2d417b95/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:52.641401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:52.686028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565332fe-5fff-4cd9-b710-f91689271f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21198d80eccc48d4bdc9f4a625eb1e11, user_id=97fa9ff620684c1b8a56044d2d417b95, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:52.688475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565332fe-5fff-4cd9-b710-f91689271f14 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:52.772705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4939c56-4860-44ec-a3fc-6b77c4932975 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:52.778894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 862/1730] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:52 2022] PUT /identity/v3/projects/21198d80eccc48d4bdc9f4a625eb1e11/users/97fa9ff620684c1b8a56044d2d417b95/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:42:52.778894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565332fe-5fff-4cd9-b710-f91689271f14 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:52.779070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c4939c56-4860-44ec-a3fc-6b77c4932975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:52.865094 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4939c56-4860-44ec-a3fc-6b77c4932975 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:52.912953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565332fe-5fff-4cd9-b710-f91689271f14 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:52.929701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956aff6-2742-4c57-b63e-7a77c2f43877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:52.929701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956aff6-2742-4c57-b63e-7a77c2f43877 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:52.929701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956aff6-2742-4c57-b63e-7a77c2f43877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:52.934148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956aff6-2742-4c57-b63e-7a77c2f43877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:52.936056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956aff6-2742-4c57-b63e-7a77c2f43877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:52.938489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956aff6-2742-4c57-b63e-7a77c2f43877 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:52.979595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10689ff-dc90-4b0c-8b77-c6ad8f7533d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:52.980094 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10689ff-dc90-4b0c-8b77-c6ad8f7533d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:52.980375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10689ff-dc90-4b0c-8b77-c6ad8f7533d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:52.980639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10689ff-dc90-4b0c-8b77-c6ad8f7533d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:52.989913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 863/1731] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:42:52.997015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956aff6-2742-4c57-b63e-7a77c2f43877 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:53.010377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10689ff-dc90-4b0c-8b77-c6ad8f7533d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:53.011817 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10689ff-dc90-4b0c-8b77-c6ad8f7533d5 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:53.022782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956aff6-2742-4c57-b63e-7a77c2f43877 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:53.030800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 869/1732] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:53.033175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467cdb2-7c6f-4841-8aeb-815a326ab3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:53.034100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467cdb2-7c6f-4841-8aeb-815a326ab3d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:53.034547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467cdb2-7c6f-4841-8aeb-815a326ab3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:53.035036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467cdb2-7c6f-4841-8aeb-815a326ab3d1 admin admin] PATH_INFO: `/v3/projects/21198d80eccc48d4bdc9f4a625eb1e11/users/97fa9ff620684c1b8a56044d2d417b95/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:53.037522 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467cdb2-7c6f-4841-8aeb-815a326ab3d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21198d80eccc48d4bdc9f4a625eb1e11, user_id=97fa9ff620684c1b8a56044d2d417b95, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:53.040926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467cdb2-7c6f-4841-8aeb-815a326ab3d1 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:53.058310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 864/1733] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:42:52 2022] PUT /identity/v3/projects/21198d80eccc48d4bdc9f4a625eb1e11/users/97fa9ff620684c1b8a56044d2d417b95/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:53.065647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89182851-9e75-49fc-83cc-f95f806d6b90 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:53.066099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89182851-9e75-49fc-83cc-f95f806d6b90 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:53.066611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89182851-9e75-49fc-83cc-f95f806d6b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:53.106863 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-89182851-9e75-49fc-83cc-f95f806d6b90 None None] MFA Rules not processed for user `97fa9ff620684c1b8a56044d2d417b95`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:53.108269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-89182851-9e75-49fc-83cc-f95f806d6b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:53.192683 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:53.310183 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 870/1734] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:53.325188 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467cdb2-7c6f-4841-8aeb-815a326ab3d1 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:53.327022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3467cdb2-7c6f-4841-8aeb-815a326ab3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:53.426039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467cdb2-7c6f-4841-8aeb-815a326ab3d1 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:53.428584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:53.431832 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe80a24-a645-4033-a9bc-db73fc6038c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:53.432642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe80a24-a645-4033-a9bc-db73fc6038c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:53.433079 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe80a24-a645-4033-a9bc-db73fc6038c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:53.436624 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe80a24-a645-4033-a9bc-db73fc6038c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:53.475132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe80a24-a645-4033-a9bc-db73fc6038c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:53.489883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe80a24-a645-4033-a9bc-db73fc6038c6 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:53.508151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 865/1735] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:53.556835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89182851-9e75-49fc-83cc-f95f806d6b90 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:53.622095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52557573-f566-4ca0-8182-ab39b8399b4c None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:53.623474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52557573-f566-4ca0-8182-ab39b8399b4c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:53.623842 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52557573-f566-4ca0-8182-ab39b8399b4c None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:53.627440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 866/1736] 10.222.0.49 () {54 vars in 912 bytes} [Fri Sep 16 05:42:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:42:53.642007 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904f47-ddba-43bf-9aee-5d00ade8502f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:53.642399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904f47-ddba-43bf-9aee-5d00ade8502f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:53.642794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904f47-ddba-43bf-9aee-5d00ade8502f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:53.716025 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c3904f47-ddba-43bf-9aee-5d00ade8502f None None] MFA Rules not processed for user `3e923be9aeed43728690d24d85dbe062`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:53.717244 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c3904f47-ddba-43bf-9aee-5d00ade8502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:53.809455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89182851-9e75-49fc-83cc-f95f806d6b90 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:53.815932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:53.817324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:53.817935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:53.819077 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] PATH_INFO: `/v3/users/4431008e78b34bc0bfc515d4125e8ae1` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:53.820506 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4431008e78b34bc0bfc515d4125e8ae1)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:53.830805 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:53.874331 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 867/1737] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:42:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:42:53.942755 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3904f47-ddba-43bf-9aee-5d00ade8502f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:53.961873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3904f47-ddba-43bf-9aee-5d00ade8502f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:53.968796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2142d-7110-4829-b22c-de7fa2f8a754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:53.970014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2142d-7110-4829-b22c-de7fa2f8a754 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:53.971404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2142d-7110-4829-b22c-de7fa2f8a754 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:53.972091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2142d-7110-4829-b22c-de7fa2f8a754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:53.990727 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2142d-7110-4829-b22c-de7fa2f8a754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:53.993237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2142d-7110-4829-b22c-de7fa2f8a754 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:54.011429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 868/1738] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:42:54.099673 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4431008e78b34bc0bfc515d4125e8ae1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:54.100588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4431008e78b34bc0bfc515d4125e8ae1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:54.176956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4431008e78b34bc0bfc515d4125e8ae1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:54.188541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4431008e78b34bc0bfc515d4125e8ae1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:54.189278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4431008e78b34bc0bfc515d4125e8ae1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:54.219544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 871/1739] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:42:53 2022] DELETE /identity/v3/users/4431008e78b34bc0bfc515d4125e8ae1 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:54.228447 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f691-38de-4b14-bfdf-560409212d54 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:54.229527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f691-38de-4b14-bfdf-560409212d54 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:54.229801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f691-38de-4b14-bfdf-560409212d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:54.289238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-3f47f691-38de-4b14-bfdf-560409212d54 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:54.295175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3f47f691-38de-4b14-bfdf-560409212d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:54.395500 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:54.400050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:54.426369 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda33c6-c978-45eb-9b9c-f611cc7ddb3c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:54.427925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 869/1740] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:54.436324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:54.437426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:54.439514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:54.440347 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] PATH_INFO: `/v3/projects/0f4504fc10b84d0aa5a5fd253a89f54f` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:54.442192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f4504fc10b84d0aa5a5fd253a89f54f)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:54.444349 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:54.458985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47f691-38de-4b14-bfdf-560409212d54 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:54.461073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3f47f691-38de-4b14-bfdf-560409212d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:54.579783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47f691-38de-4b14-bfdf-560409212d54 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:54.611622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f4504fc10b84d0aa5a5fd253a89f54f', 'request_id': 'req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:54.624008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f4504fc10b84d0aa5a5fd253a89f54f', 'request_id': 'req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:54.635992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f4504fc10b84d0aa5a5fd253a89f54f', 'request_id': 'req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:54.681503 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c176fbc-2db0-4c7d-9248-280e9c8f7bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:54.682453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c176fbc-2db0-4c7d-9248-280e9c8f7bae service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:54.682812 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c176fbc-2db0-4c7d-9248-280e9c8f7bae service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:54.683190 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c176fbc-2db0-4c7d-9248-280e9c8f7bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:54.741408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] The token cache is being invalidate because project 0f4504fc10b84d0aa5a5fd253a89f54f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:42:54.741916 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:54.743752 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:54.746484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 872/1741] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:42:54 2022] DELETE /identity/v3/projects/0f4504fc10b84d0aa5a5fd253a89f54f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:54.751753 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:54.757969 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:54.772120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:54.894555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c176fbc-2db0-4c7d-9248-280e9c8f7bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:54.897475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c176fbc-2db0-4c7d-9248-280e9c8f7bae service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:54.937834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 870/1742] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:42:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:42:54.939228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e23fe-8005-4553-bf48-f0ce2811ef60 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:54.939818 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e23fe-8005-4553-bf48-f0ce2811ef60 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:54.940429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e23fe-8005-4553-bf48-f0ce2811ef60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:55.025624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0e8b8-07e7-4ca9-8f60-a13fa33271bb admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:55.039825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c61e23fe-8005-4553-bf48-f0ce2811ef60 None None] MFA Rules not processed for user `edd619bf865645fab0460d3aa847a734`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:55.039825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c61e23fe-8005-4553-bf48-f0ce2811ef60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:55.039991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e940b-1b54-4730-8651-9e5cbd161ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:55.041586 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e940b-1b54-4730-8651-9e5cbd161ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:55.042198 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e940b-1b54-4730-8651-9e5cbd161ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:55.042926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e940b-1b54-4730-8651-9e5cbd161ec2 admin admin] PATH_INFO: `/v3/users/e65f14431dc94fdf8768655d90eb9219` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:55.047536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e940b-1b54-4730-8651-9e5cbd161ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e65f14431dc94fdf8768655d90eb9219)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:55.058713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7e940b-1b54-4730-8651-9e5cbd161ec2 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:55.231964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 871/1743] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:55.340369 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2e7e940b-1b54-4730-8651-9e5cbd161ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e65f14431dc94fdf8768655d90eb9219'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:55.341450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2e7e940b-1b54-4730-8651-9e5cbd161ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e65f14431dc94fdf8768655d90eb9219'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:55.438430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2e7e940b-1b54-4730-8651-9e5cbd161ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e65f14431dc94fdf8768655d90eb9219'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:55.462587 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2e7e940b-1b54-4730-8651-9e5cbd161ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e65f14431dc94fdf8768655d90eb9219'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:55.463480 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2e7e940b-1b54-4730-8651-9e5cbd161ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e65f14431dc94fdf8768655d90eb9219'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:55.535771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:55.542461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 873/1744] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:42:54 2022] DELETE /identity/v3/users/e65f14431dc94fdf8768655d90eb9219 => generated 0 bytes in 791 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:55.937667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:56.138015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:56.233023 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e23fe-8005-4553-bf48-f0ce2811ef60 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:56.236127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c61e23fe-8005-4553-bf48-f0ce2811ef60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:56.254687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e23fe-8005-4553-bf48-f0ce2811ef60 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:56.445635 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:56.446536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:56.447366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:56.447784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] PATH_INFO: `/v3/projects/936662ef12c046d98873d6efccaa17ab` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:56.449201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=936662ef12c046d98873d6efccaa17ab)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:56.451390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:56.736056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '936662ef12c046d98873d6efccaa17ab', 'request_id': 'req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:56.753383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '936662ef12c046d98873d6efccaa17ab', 'request_id': 'req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:56.783821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '936662ef12c046d98873d6efccaa17ab', 'request_id': 'req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:56.838711 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bb335-7641-41be-9147-f6cd1fca0fac None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:56.841906 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bb335-7641-41be-9147-f6cd1fca0fac None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:56.842395 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bb335-7641-41be-9147-f6cd1fca0fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:56.901440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] The token cache is being invalidate because project 936662ef12c046d98873d6efccaa17ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:42:56.902132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:56.903873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:56.916944 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 872/1745] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:42:56 2022] DELETE /identity/v3/projects/936662ef12c046d98873d6efccaa17ab => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:56.919889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:56.928898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:56.951917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-827bb335-7641-41be-9147-f6cd1fca0fac None None] MFA Rules not processed for user `c205ee9f85924da79b870648698caf64`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:56.954362 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:56.958962 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-827bb335-7641-41be-9147-f6cd1fca0fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:57.085844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f46b5-acd7-4b16-be6d-5430dc5655ee admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:57.093770 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:57.094462 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:57.098806 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:57.098806 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] PATH_INFO: `/v3/users/97fa9ff620684c1b8a56044d2d417b95` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:57.104926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97fa9ff620684c1b8a56044d2d417b95)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:57.107962 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:57.141715 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 874/1746] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:57.173308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bb335-7641-41be-9147-f6cd1fca0fac None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:57.176502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-827bb335-7641-41be-9147-f6cd1fca0fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:57.200626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bb335-7641-41be-9147-f6cd1fca0fac None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:57.217707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:57.217707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:57.218538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:57.218538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] PATH_INFO: `/v3/projects/be76f741eb4d43ff95ef7ca86d6cf594` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:57.220416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=be76f741eb4d43ff95ef7ca86d6cf594)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:57.225278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:57.363926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97fa9ff620684c1b8a56044d2d417b95'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:57.366581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97fa9ff620684c1b8a56044d2d417b95'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:57.461472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97fa9ff620684c1b8a56044d2d417b95'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:57.462342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97fa9ff620684c1b8a56044d2d417b95'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:57.474105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97fa9ff620684c1b8a56044d2d417b95'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:57.482203 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be76f741eb4d43ff95ef7ca86d6cf594', 'request_id': 'req-c6215138-e49e-437a-88f5-4b69328f2b1c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:57.511907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be76f741eb4d43ff95ef7ca86d6cf594', 'request_id': 'req-c6215138-e49e-437a-88f5-4b69328f2b1c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:57.530203 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be76f741eb4d43ff95ef7ca86d6cf594', 'request_id': 'req-c6215138-e49e-437a-88f5-4b69328f2b1c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:57.562923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 873/1747] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:42:56 2022] DELETE /identity/v3/users/97fa9ff620684c1b8a56044d2d417b95 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:57.621730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] The token cache is being invalidate because project be76f741eb4d43ff95ef7ca86d6cf594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:42:57.622290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:57.623793 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:57.626338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 875/1748] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:42:57 2022] DELETE /identity/v3/projects/be76f741eb4d43ff95ef7ca86d6cf594 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:42:57.630459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:57.632154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:57.815314 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9278e6-bc02-4321-8318-be41ebc09319 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:57.820958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a8cbc-4a73-4494-83fe-0db35e5ee68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:57.821814 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a8cbc-4a73-4494-83fe-0db35e5ee68a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:57.822291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a8cbc-4a73-4494-83fe-0db35e5ee68a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:57.822705 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a8cbc-4a73-4494-83fe-0db35e5ee68a admin admin] PATH_INFO: `/v3/users/fba0eb27c84640709d417d60804aa951` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:57.824051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a8cbc-4a73-4494-83fe-0db35e5ee68a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba0eb27c84640709d417d60804aa951)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:57.826227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030a8cbc-4a73-4494-83fe-0db35e5ee68a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:58.094026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:58.124131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-030a8cbc-4a73-4494-83fe-0db35e5ee68a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba0eb27c84640709d417d60804aa951'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:58.125345 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-030a8cbc-4a73-4494-83fe-0db35e5ee68a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba0eb27c84640709d417d60804aa951'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:58.346661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-030a8cbc-4a73-4494-83fe-0db35e5ee68a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba0eb27c84640709d417d60804aa951'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:58.348508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-030a8cbc-4a73-4494-83fe-0db35e5ee68a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba0eb27c84640709d417d60804aa951'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:58.356860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-030a8cbc-4a73-4494-83fe-0db35e5ee68a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba0eb27c84640709d417d60804aa951'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:58.422281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 874/1749] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:42:57 2022] DELETE /identity/v3/users/fba0eb27c84640709d417d60804aa951 => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:58.642766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:58.647477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:58.856049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6215138-e49e-437a-88f5-4b69328f2b1c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:59.009520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c911a4c-1cab-434c-858a-162d1c6f5924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:59.012295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c911a4c-1cab-434c-858a-162d1c6f5924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:59.012621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c911a4c-1cab-434c-858a-162d1c6f5924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:59.013018 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c911a4c-1cab-434c-858a-162d1c6f5924 admin admin] PATH_INFO: `/v3/projects/21198d80eccc48d4bdc9f4a625eb1e11` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:59.016005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c911a4c-1cab-434c-858a-162d1c6f5924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21198d80eccc48d4bdc9f4a625eb1e11)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:59.020516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c911a4c-1cab-434c-858a-162d1c6f5924 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:59.212267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2c911a4c-1cab-434c-858a-162d1c6f5924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21198d80eccc48d4bdc9f4a625eb1e11', 'request_id': 'req-2c911a4c-1cab-434c-858a-162d1c6f5924'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:59.218173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:42:59.223716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2c911a4c-1cab-434c-858a-162d1c6f5924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21198d80eccc48d4bdc9f4a625eb1e11', 'request_id': 'req-2c911a4c-1cab-434c-858a-162d1c6f5924'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:59.235419 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2c911a4c-1cab-434c-858a-162d1c6f5924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21198d80eccc48d4bdc9f4a625eb1e11', 'request_id': 'req-2c911a4c-1cab-434c-858a-162d1c6f5924'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:59.357285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2c911a4c-1cab-434c-858a-162d1c6f5924 admin admin] The token cache is being invalidate because project 21198d80eccc48d4bdc9f4a625eb1e11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:42:59.357285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2c911a4c-1cab-434c-858a-162d1c6f5924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:59.357285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2c911a4c-1cab-434c-858a-162d1c6f5924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:42:59.359244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 876/1750] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:42:58 2022] DELETE /identity/v3/projects/21198d80eccc48d4bdc9f4a625eb1e11 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:42:59.365014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3ae8e-058e-42f3-a3da-eadc8ef47c20 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:59.365271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3ae8e-058e-42f3-a3da-eadc8ef47c20 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:59.365545 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3ae8e-058e-42f3-a3da-eadc8ef47c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:59.469071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-80a3ae8e-058e-42f3-a3da-eadc8ef47c20 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:59.483319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-80a3ae8e-058e-42f3-a3da-eadc8ef47c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:59.593086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffc6c5-94a0-4c96-a0dc-0f723bf81895 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:59.593674 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffc6c5-94a0-4c96-a0dc-0f723bf81895 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:59.594135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffc6c5-94a0-4c96-a0dc-0f723bf81895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:59.717148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 875/1751] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:59.722958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0774c-ef36-430b-bd9b-b0e90ebfa93b None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:59.723417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0774c-ef36-430b-bd9b-b0e90ebfa93b None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:59.723699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0774c-ef36-430b-bd9b-b0e90ebfa93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:59.737597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-71ffc6c5-94a0-4c96-a0dc-0f723bf81895 None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:59.744753 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-71ffc6c5-94a0-4c96-a0dc-0f723bf81895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:59.782630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-f6a0774c-ef36-430b-bd9b-b0e90ebfa93b None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:42:59.790408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f6a0774c-ef36-430b-bd9b-b0e90ebfa93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:59.838833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 876/1752] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:42:59.847635 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0774c-ef36-430b-bd9b-b0e90ebfa93b None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:59.896501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0774c-ef36-430b-bd9b-b0e90ebfa93b None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:59.925153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa382da-f90c-49c5-aaae-ff2e02a390e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:42:59.926521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa382da-f90c-49c5-aaae-ff2e02a390e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:42:59.927183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa382da-f90c-49c5-aaae-ff2e02a390e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:42:59.927559 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa382da-f90c-49c5-aaae-ff2e02a390e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:42:59.928322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa382da-f90c-49c5-aaae-ff2e02a390e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:42:59.929799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa382da-f90c-49c5-aaae-ff2e02a390e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:42:59.938464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa382da-f90c-49c5-aaae-ff2e02a390e6 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:42:59.949857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 877/1753] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:42:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:42:59.959776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffc6c5-94a0-4c96-a0dc-0f723bf81895 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:42:59.971420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-71ffc6c5-94a0-4c96-a0dc-0f723bf81895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:42:59.989212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 877/1754] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:42:59.991724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffc6c5-94a0-4c96-a0dc-0f723bf81895 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:42:59.993723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa382da-f90c-49c5-aaae-ff2e02a390e6 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:00.007528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:00.009974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:00.010430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:00.012355 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] PATH_INFO: `/v3/projects/d25a4129bfd24e0e9128c1e7f6a2b6cd` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:00.014756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d25a4129bfd24e0e9128c1e7f6a2b6cd)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:00.017633 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa382da-f90c-49c5-aaae-ff2e02a390e6 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:00.019949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:00.032074 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b8a7a-759c-4857-b6d1-fbff1ac789f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:00.033452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b8a7a-759c-4857-b6d1-fbff1ac789f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:00.034257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b8a7a-759c-4857-b6d1-fbff1ac789f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:00.035450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b8a7a-759c-4857-b6d1-fbff1ac789f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:00.036767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b8a7a-759c-4857-b6d1-fbff1ac789f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:00.042584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b8a7a-759c-4857-b6d1-fbff1ac789f1 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:00.236714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 878/1755] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:42:59 2022] POST /identity/v3/projects => generated 377 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:00.242610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b8a7a-759c-4857-b6d1-fbff1ac789f1 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:00.279174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd25a4129bfd24e0e9128c1e7f6a2b6cd', 'request_id': 'req-69058533-94c0-449c-9e76-9bc8646e0fe1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:00.303064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd25a4129bfd24e0e9128c1e7f6a2b6cd', 'request_id': 'req-69058533-94c0-449c-9e76-9bc8646e0fe1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:00.314345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd25a4129bfd24e0e9128c1e7f6a2b6cd', 'request_id': 'req-69058533-94c0-449c-9e76-9bc8646e0fe1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:00.375235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b8a7a-759c-4857-b6d1-fbff1ac789f1 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:00.480027 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] The token cache is being invalidate because project d25a4129bfd24e0e9128c1e7f6a2b6cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:00.480435 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:00.482143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:00.490209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 878/1756] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:42:59 2022] DELETE /identity/v3/projects/d25a4129bfd24e0e9128c1e7f6a2b6cd => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:00.490941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:00.497789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:00.560567 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31394cc6-af8e-47d6-83f4-d16c9d899385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:00.561207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31394cc6-af8e-47d6-83f4-d16c9d899385 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:00.563762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31394cc6-af8e-47d6-83f4-d16c9d899385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:00.563762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31394cc6-af8e-47d6-83f4-d16c9d899385 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:00.563762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31394cc6-af8e-47d6-83f4-d16c9d899385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:00.565441 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31394cc6-af8e-47d6-83f4-d16c9d899385 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:00.679891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 879/1757] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:00 2022] POST /identity/v3/users => generated 289 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:00.682344 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67104313-9a67-4400-9db6-9c178e9b66a2 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:00.682790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67104313-9a67-4400-9db6-9c178e9b66a2 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:00.683919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67104313-9a67-4400-9db6-9c178e9b66a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:00.725258 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-67104313-9a67-4400-9db6-9c178e9b66a2 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:00.726728 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-67104313-9a67-4400-9db6-9c178e9b66a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:00.746103 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 880/1758] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:00.746103 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67104313-9a67-4400-9db6-9c178e9b66a2 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:00.746103 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-67104313-9a67-4400-9db6-9c178e9b66a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:00.769930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67104313-9a67-4400-9db6-9c178e9b66a2 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:00.778891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79360b3-cc09-4120-8f88-0aa9f5d52e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:00.779919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79360b3-cc09-4120-8f88-0aa9f5d52e04 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:00.780511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79360b3-cc09-4120-8f88-0aa9f5d52e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:00.781005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79360b3-cc09-4120-8f88-0aa9f5d52e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:00.781960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79360b3-cc09-4120-8f88-0aa9f5d52e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:00.784000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79360b3-cc09-4120-8f88-0aa9f5d52e04 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:00.797782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 881/1759] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:00.799793 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad17e0-275f-4079-8c62-bd71ffa47162 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:00.800308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad17e0-275f-4079-8c62-bd71ffa47162 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:00.800862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad17e0-275f-4079-8c62-bd71ffa47162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:00.869296 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-bcad17e0-275f-4079-8c62-bd71ffa47162 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:00.872192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:00.875949 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-bcad17e0-275f-4079-8c62-bd71ffa47162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:00.884961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 882/1760] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:00.885783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad17e0-275f-4079-8c62-bd71ffa47162 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:00.891949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69058533-94c0-449c-9e76-9bc8646e0fe1 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:00.898447 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec009a-78ff-439a-9292-822224bb86c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:00.899082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec009a-78ff-439a-9292-822224bb86c2 service swift] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:00.904034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec009a-78ff-439a-9292-822224bb86c2 service swift] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:00.904034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec009a-78ff-439a-9292-822224bb86c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:00.907093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9aec009a-78ff-439a-9292-822224bb86c2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:00.950405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad17e0-275f-4079-8c62-bd71ffa47162 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:00.959880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e68c9-45dd-494d-94be-75bd8eb4d79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:00.961398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e68c9-45dd-494d-94be-75bd8eb4d79a admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:00.961398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e68c9-45dd-494d-94be-75bd8eb4d79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:00.961637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e68c9-45dd-494d-94be-75bd8eb4d79a admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:00.962488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e68c9-45dd-494d-94be-75bd8eb4d79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:00.964161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e68c9-45dd-494d-94be-75bd8eb4d79a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:00.977609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 883/1761] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:00.979031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e68c9-45dd-494d-94be-75bd8eb4d79a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:00.996057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e68c9-45dd-494d-94be-75bd8eb4d79a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:01.007010 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9345df7-28da-4335-acb8-0e426be965a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:01.009041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9345df7-28da-4335-acb8-0e426be965a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:01.009041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9345df7-28da-4335-acb8-0e426be965a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:01.009041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9345df7-28da-4335-acb8-0e426be965a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:01.010647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9345df7-28da-4335-acb8-0e426be965a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:43:01.012060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9345df7-28da-4335-acb8-0e426be965a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:01.020961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9345df7-28da-4335-acb8-0e426be965a4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:01.043730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 884/1762] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:43:01.045164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9345df7-28da-4335-acb8-0e426be965a4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:01.060411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9345df7-28da-4335-acb8-0e426be965a4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:01.069036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e2d52-ad4c-404c-9543-20981847460b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:01.069036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e2d52-ad4c-404c-9543-20981847460b admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:01.069036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e2d52-ad4c-404c-9543-20981847460b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:01.069489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e2d52-ad4c-404c-9543-20981847460b admin admin] PATH_INFO: `/v3/projects/e6243987d41741bfb728405997d57536/users/4b19d6c7585e42e59300a9ba395f8759/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:01.098389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e2d52-ad4c-404c-9543-20981847460b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6243987d41741bfb728405997d57536, user_id=4b19d6c7585e42e59300a9ba395f8759, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:01.106414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e2d52-ad4c-404c-9543-20981847460b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:01.159435 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec009a-78ff-439a-9292-822224bb86c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:01.166648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec009a-78ff-439a-9292-822224bb86c2 service swift] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:01.205188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 879/1763] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:43:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 716 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:43:01.205188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec009a-78ff-439a-9292-822224bb86c2 service swift] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:01.242248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 885/1764] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:01 2022] PUT /identity/v3/projects/e6243987d41741bfb728405997d57536/users/4b19d6c7585e42e59300a9ba395f8759/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:01.258356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47e732-28ea-4205-8a90-e9e4bba2c04e None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:01.258650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47e732-28ea-4205-8a90-e9e4bba2c04e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:01.258925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47e732-28ea-4205-8a90-e9e4bba2c04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:01.278471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec009a-78ff-439a-9292-822224bb86c2 service swift] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:01.371713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-aa47e732-28ea-4205-8a90-e9e4bba2c04e None None] MFA Rules not processed for user `4b19d6c7585e42e59300a9ba395f8759`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:01.380733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-aa47e732-28ea-4205-8a90-e9e4bba2c04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:01.539410 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f1611-5844-4cb9-856c-fb1272d9feaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:01.542872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f1611-5844-4cb9-856c-fb1272d9feaa admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:01.546187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f1611-5844-4cb9-856c-fb1272d9feaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:01.546187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f1611-5844-4cb9-856c-fb1272d9feaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:01.550059 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f1611-5844-4cb9-856c-fb1272d9feaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:01.557183 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7f1611-5844-4cb9-856c-fb1272d9feaa admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:01.562718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 886/1765] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:01.576286 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47e732-28ea-4205-8a90-e9e4bba2c04e None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:01.650114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47e732-28ea-4205-8a90-e9e4bba2c04e None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:01.703295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797440b4-1d05-4cc3-a18f-6a83a23f44f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:01.709471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797440b4-1d05-4cc3-a18f-6a83a23f44f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:01.709932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797440b4-1d05-4cc3-a18f-6a83a23f44f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:01.713723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:01.714600 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797440b4-1d05-4cc3-a18f-6a83a23f44f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:01.718868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797440b4-1d05-4cc3-a18f-6a83a23f44f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:01.721472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 880/1766] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:01 2022] POST /identity/v3/projects => generated 371 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:01.726660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f1611-5844-4cb9-856c-fb1272d9feaa admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:01.734294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797440b4-1d05-4cc3-a18f-6a83a23f44f7 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:01.917028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 887/1767] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:01 2022] POST /identity/v3/projects => generated 377 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:01.918557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797440b4-1d05-4cc3-a18f-6a83a23f44f7 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:02.103482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797440b4-1d05-4cc3-a18f-6a83a23f44f7 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:02.118750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c896b1-7177-412f-93a1-5cc13f8435c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:02.119402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c896b1-7177-412f-93a1-5cc13f8435c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:02.119683 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c896b1-7177-412f-93a1-5cc13f8435c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:02.119958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c896b1-7177-412f-93a1-5cc13f8435c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:02.120607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c896b1-7177-412f-93a1-5cc13f8435c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:02.124060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c896b1-7177-412f-93a1-5cc13f8435c8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:02.218749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f1611-5844-4cb9-856c-fb1272d9feaa admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:02.261488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 888/1768] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:01 2022] POST /identity/v3/users => generated 289 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:43:02.262696 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c896b1-7177-412f-93a1-5cc13f8435c8 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:02.265223 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6cda0-f0cb-498a-b1ac-d9afa3a54027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:02.265880 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6cda0-f0cb-498a-b1ac-d9afa3a54027 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:02.266163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6cda0-f0cb-498a-b1ac-d9afa3a54027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:02.266435 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6cda0-f0cb-498a-b1ac-d9afa3a54027 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:02.283162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6cda0-f0cb-498a-b1ac-d9afa3a54027 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:02.285797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6cda0-f0cb-498a-b1ac-d9afa3a54027 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:02.286546 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c896b1-7177-412f-93a1-5cc13f8435c8 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:02.301766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9cb91-b21c-4b8a-8fe2-430c134b8fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:02.303183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9cb91-b21c-4b8a-8fe2-430c134b8fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:02.303897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9cb91-b21c-4b8a-8fe2-430c134b8fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:02.305116 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9cb91-b21c-4b8a-8fe2-430c134b8fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:02.306418 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe9cb91-b21c-4b8a-8fe2-430c134b8fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:02.309031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe9cb91-b21c-4b8a-8fe2-430c134b8fe8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:02.355553 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 889/1769] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:43:02.361588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9cb91-b21c-4b8a-8fe2-430c134b8fe8 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:02.420519 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9cb91-b21c-4b8a-8fe2-430c134b8fe8 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:02.445458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e81d94-d7c9-4a12-87d1-e0cd8917d463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:02.447184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e81d94-d7c9-4a12-87d1-e0cd8917d463 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:02.449223 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e81d94-d7c9-4a12-87d1-e0cd8917d463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:02.451275 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e81d94-d7c9-4a12-87d1-e0cd8917d463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:02.453722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e81d94-d7c9-4a12-87d1-e0cd8917d463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:02.459734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e81d94-d7c9-4a12-87d1-e0cd8917d463 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:02.507929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 881/1770] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:01 2022] POST /identity/v3/users => generated 286 bytes in 779 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:02.508141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 890/1771] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:02.508222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776eb429-eb04-40c3-a752-9b9ea36c623c None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:02.508222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776eb429-eb04-40c3-a752-9b9ea36c623c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:02.508345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776eb429-eb04-40c3-a752-9b9ea36c623c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:02.508859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e81d94-d7c9-4a12-87d1-e0cd8917d463 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:02.536195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e81d94-d7c9-4a12-87d1-e0cd8917d463 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:02.545093 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501df17-3155-4308-b7aa-5ce101d0cd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:02.546653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501df17-3155-4308-b7aa-5ce101d0cd25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:02.549792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501df17-3155-4308-b7aa-5ce101d0cd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:02.550504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501df17-3155-4308-b7aa-5ce101d0cd25 admin admin] PATH_INFO: `/v3/projects/de0792ab075a4239ad92ff4d454d7673/users/fb2778bf13f94f92bbb34071d826126d/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:02.580499 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501df17-3155-4308-b7aa-5ce101d0cd25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de0792ab075a4239ad92ff4d454d7673, user_id=fb2778bf13f94f92bbb34071d826126d, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:02.586665 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b501df17-3155-4308-b7aa-5ce101d0cd25 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:02.609480 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:02.620772 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-776eb429-eb04-40c3-a752-9b9ea36c623c None None] MFA Rules not processed for user `3e923be9aeed43728690d24d85dbe062`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:02.620772 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-776eb429-eb04-40c3-a752-9b9ea36c623c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:02.715476 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 891/1772] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:02 2022] PUT /identity/v3/projects/de0792ab075a4239ad92ff4d454d7673/users/fb2778bf13f94f92bbb34071d826126d/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:02.717060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501df17-3155-4308-b7aa-5ce101d0cd25 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:02.833575 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 882/1773] 10.222.0.15 () {58 vars in 1015 bytes} [Fri Sep 16 05:43:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:02.835453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776eb429-eb04-40c3-a752-9b9ea36c623c None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:02.841604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-776eb429-eb04-40c3-a752-9b9ea36c623c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:02.902363 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776eb429-eb04-40c3-a752-9b9ea36c623c None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:02.919423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:02.940271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501df17-3155-4308-b7aa-5ce101d0cd25 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:02.948863 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:02.949521 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:02.949960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:02.950173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] PATH_INFO: `/v3/users/89299b0f46da4796954983b30a37cf2e` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:02.958119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=89299b0f46da4796954983b30a37cf2e)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:02.960158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:02.961605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c297cd-4f2b-404b-8f39-9baab574fc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:02.964893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c297cd-4f2b-404b-8f39-9baab574fc00 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:02.965635 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c297cd-4f2b-404b-8f39-9baab574fc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:02.965954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c297cd-4f2b-404b-8f39-9baab574fc00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:02.971669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c297cd-4f2b-404b-8f39-9baab574fc00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:02.985693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c297cd-4f2b-404b-8f39-9baab574fc00 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:03.039352 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 892/1774] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:03.045094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158e895-5a02-4422-8e0d-ec9032a475f3 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:03.046818 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158e895-5a02-4422-8e0d-ec9032a475f3 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:03.048076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158e895-5a02-4422-8e0d-ec9032a475f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:03.145946 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a158e895-5a02-4422-8e0d-ec9032a475f3 None None] MFA Rules not processed for user `fb2778bf13f94f92bbb34071d826126d`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:03.153920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a158e895-5a02-4422-8e0d-ec9032a475f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:03.520062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 893/1775] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:03.520062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158e895-5a02-4422-8e0d-ec9032a475f3 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:03.520062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a158e895-5a02-4422-8e0d-ec9032a475f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:03.524660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89299b0f46da4796954983b30a37cf2e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:03.524660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89299b0f46da4796954983b30a37cf2e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:03.526204 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158e895-5a02-4422-8e0d-ec9032a475f3 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:03.526204 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6a2e8-379f-4006-87a6-fe55d6f52280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:03.526204 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6a2e8-379f-4006-87a6-fe55d6f52280 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:03.526204 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6a2e8-379f-4006-87a6-fe55d6f52280 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:03.527495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6a2e8-379f-4006-87a6-fe55d6f52280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:03.527495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6a2e8-379f-4006-87a6-fe55d6f52280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:03.527495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6a2e8-379f-4006-87a6-fe55d6f52280 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:03.527495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 894/1776] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:03.527495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6a2e8-379f-4006-87a6-fe55d6f52280 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:03.527495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6a2e8-379f-4006-87a6-fe55d6f52280 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:03.531573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe7b0f-e56a-4879-991e-f955212eaaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:03.531573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe7b0f-e56a-4879-991e-f955212eaaa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:03.531573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe7b0f-e56a-4879-991e-f955212eaaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:03.531573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe7b0f-e56a-4879-991e-f955212eaaa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:03.532289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe7b0f-e56a-4879-991e-f955212eaaa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:03.532289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe7b0f-e56a-4879-991e-f955212eaaa2 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:03.532289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 895/1777] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:03.532289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe7b0f-e56a-4879-991e-f955212eaaa2 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:03.556792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe7b0f-e56a-4879-991e-f955212eaaa2 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:03.576514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682034f-ae9c-4a7c-83ef-297ed0b676e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:03.579147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682034f-ae9c-4a7c-83ef-297ed0b676e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:03.609087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682034f-ae9c-4a7c-83ef-297ed0b676e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:03.609087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682034f-ae9c-4a7c-83ef-297ed0b676e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:03.609087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682034f-ae9c-4a7c-83ef-297ed0b676e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:03.609087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682034f-ae9c-4a7c-83ef-297ed0b676e2 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:03.673495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:03.695213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89299b0f46da4796954983b30a37cf2e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:03.695213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89299b0f46da4796954983b30a37cf2e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:03.695213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89299b0f46da4796954983b30a37cf2e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:03.746113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 883/1778] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:02 2022] DELETE /identity/v3/users/89299b0f46da4796954983b30a37cf2e => generated 0 bytes in 909 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:03.747115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:03.754173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:03.855423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 896/1779] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:03 2022] POST /identity/v3/projects => generated 377 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:03.855423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682034f-ae9c-4a7c-83ef-297ed0b676e2 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:03.856808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a682034f-ae9c-4a7c-83ef-297ed0b676e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:03.952569 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:03.969781 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8306ee1-2318-4530-a6aa-296a6a57ebcb admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:04.096994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682034f-ae9c-4a7c-83ef-297ed0b676e2 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:04.139358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:04.141165 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33859d8-2ed0-4bf9-aeae-cd9a772ae42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:04.142142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33859d8-2ed0-4bf9-aeae-cd9a772ae42e admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:04.142641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33859d8-2ed0-4bf9-aeae-cd9a772ae42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:04.143328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33859d8-2ed0-4bf9-aeae-cd9a772ae42e admin admin] PATH_INFO: `/v3/projects/9c4cc7113aa341a5bc8e40d03d2026e6/users/b29bb91e157048af8fea6eead061bd1d/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:04.145361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:04.145736 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:04.146171 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] PATH_INFO: `/v3/users/499c76edefeb47589181acceed0db46c` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:04.148458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] RBAC: Authorizing `identity:delete_user(user_id=499c76edefeb47589181acceed0db46c)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:04.150708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:04.197375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33859d8-2ed0-4bf9-aeae-cd9a772ae42e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4cc7113aa341a5bc8e40d03d2026e6, user_id=b29bb91e157048af8fea6eead061bd1d, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:04.200427 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33859d8-2ed0-4bf9-aeae-cd9a772ae42e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:04.341735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 884/1780] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:03 2022] PUT /identity/v3/projects/9c4cc7113aa341a5bc8e40d03d2026e6/users/b29bb91e157048af8fea6eead061bd1d/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:04.343069 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83803e-0d8c-4cea-b7b8-0aca4155d835 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:04.343451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83803e-0d8c-4cea-b7b8-0aca4155d835 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:04.343861 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83803e-0d8c-4cea-b7b8-0aca4155d835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:04.384379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '499c76edefeb47589181acceed0db46c'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:04.386528 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '499c76edefeb47589181acceed0db46c'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:04.407412 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-1b83803e-0d8c-4cea-b7b8-0aca4155d835 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:04.410303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1b83803e-0d8c-4cea-b7b8-0aca4155d835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:04.490267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '499c76edefeb47589181acceed0db46c'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:04.490267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '499c76edefeb47589181acceed0db46c'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:04.490561 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '499c76edefeb47589181acceed0db46c'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:04.547636 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 897/1781] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:03 2022] DELETE /identity/v3/users/499c76edefeb47589181acceed0db46c => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:04.548560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:04.550727 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:04.617678 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:04.634629 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 885/1782] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:04.636191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280336c-611e-4cd3-bef6-fe7c786e9039 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:04.636470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280336c-611e-4cd3-bef6-fe7c786e9039 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:04.636755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280336c-611e-4cd3-bef6-fe7c786e9039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:04.666761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:04.694057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ed579-b4b8-4c6a-ad6a-8c448071081b admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:04.700389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-5280336c-611e-4cd3-bef6-fe7c786e9039 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:04.707092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5280336c-611e-4cd3-bef6-fe7c786e9039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:04.716366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53868cea-6c73-42b6-991d-48b60112ea37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:04.716366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53868cea-6c73-42b6-991d-48b60112ea37 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:04.716366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53868cea-6c73-42b6-991d-48b60112ea37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:04.716366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53868cea-6c73-42b6-991d-48b60112ea37 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:04.716892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53868cea-6c73-42b6-991d-48b60112ea37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:04.716892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53868cea-6c73-42b6-991d-48b60112ea37 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:04.733044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 886/1783] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:04.733044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e66d6-b70f-4901-868c-3efaf5cddec8 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:04.733044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e66d6-b70f-4901-868c-3efaf5cddec8 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:04.733044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e66d6-b70f-4901-868c-3efaf5cddec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:04.781918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0f1e66d6-b70f-4901-868c-3efaf5cddec8 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:04.784454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0f1e66d6-b70f-4901-868c-3efaf5cddec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:04.811497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 887/1784] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:04.811497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e66d6-b70f-4901-868c-3efaf5cddec8 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:04.817302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0f1e66d6-b70f-4901-868c-3efaf5cddec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:04.851693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 898/1785] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:04 2022] POST /identity/v3/users => generated 295 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:04.855384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53868cea-6c73-42b6-991d-48b60112ea37 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:04.870982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53868cea-6c73-42b6-991d-48b60112ea37 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:04.882703 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211532ff-6f18-4f84-988c-7ba70c0e3a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:04.884691 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211532ff-6f18-4f84-988c-7ba70c0e3a49 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:04.886102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211532ff-6f18-4f84-988c-7ba70c0e3a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:04.886618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211532ff-6f18-4f84-988c-7ba70c0e3a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:04.888378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211532ff-6f18-4f84-988c-7ba70c0e3a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:04.891157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211532ff-6f18-4f84-988c-7ba70c0e3a49 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:04.918481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 899/1786] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:04.918481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211532ff-6f18-4f84-988c-7ba70c0e3a49 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:04.925296 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-211532ff-6f18-4f84-988c-7ba70c0e3a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:05.030426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e66d6-b70f-4901-868c-3efaf5cddec8 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:05.043291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a844591-9492-43d2-b0a9-1d7650f22fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:05.044249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a844591-9492-43d2-b0a9-1d7650f22fea service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:05.044530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a844591-9492-43d2-b0a9-1d7650f22fea service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:05.044801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a844591-9492-43d2-b0a9-1d7650f22fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:05.080454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a844591-9492-43d2-b0a9-1d7650f22fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:05.082636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a844591-9492-43d2-b0a9-1d7650f22fea service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:05.115936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 888/1787] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:05.117466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a844591-9492-43d2-b0a9-1d7650f22fea service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:05.134240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211532ff-6f18-4f84-988c-7ba70c0e3a49 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:05.145799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388cfe30-ef4a-4d51-9ffb-844ed19eb569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:05.150775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cfe30-ef4a-4d51-9ffb-844ed19eb569 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:05.151562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cfe30-ef4a-4d51-9ffb-844ed19eb569 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:05.155981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388cfe30-ef4a-4d51-9ffb-844ed19eb569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:05.162025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a844591-9492-43d2-b0a9-1d7650f22fea service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:05.180674 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2952d-d6fd-495b-9eb1-3c1149fb6aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:05.190854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2952d-d6fd-495b-9eb1-3c1149fb6aca admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:05.190854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2952d-d6fd-495b-9eb1-3c1149fb6aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:05.190854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2952d-d6fd-495b-9eb1-3c1149fb6aca admin admin] PATH_INFO: `/v3/projects/e82e030b88414e95b3883f160eacc901/users/625e9c115dac44b1b790775ca0dd7211/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:05.191150 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388cfe30-ef4a-4d51-9ffb-844ed19eb569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:05.198953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388cfe30-ef4a-4d51-9ffb-844ed19eb569 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:05.232451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:05.246205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2952d-d6fd-495b-9eb1-3c1149fb6aca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e82e030b88414e95b3883f160eacc901, user_id=625e9c115dac44b1b790775ca0dd7211, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:05.247629 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 900/1788] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:05.248574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388cfe30-ef4a-4d51-9ffb-844ed19eb569 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:05.252875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2952d-d6fd-495b-9eb1-3c1149fb6aca admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:05.256562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-388cfe30-ef4a-4d51-9ffb-844ed19eb569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:05.290936 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388cfe30-ef4a-4d51-9ffb-844ed19eb569 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:05.310093 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed11e22-8aef-4196-ac30-3b3e4fb76aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:05.310762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed11e22-8aef-4196-ac30-3b3e4fb76aee service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:05.311047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed11e22-8aef-4196-ac30-3b3e4fb76aee service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:05.311442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed11e22-8aef-4196-ac30-3b3e4fb76aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:05.335615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 889/1789] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:05 2022] PUT /identity/v3/projects/e82e030b88414e95b3883f160eacc901/users/625e9c115dac44b1b790775ca0dd7211/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:05.339931 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2952d-d6fd-495b-9eb1-3c1149fb6aca admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:05.354085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed11e22-8aef-4196-ac30-3b3e4fb76aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:05.359430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed11e22-8aef-4196-ac30-3b3e4fb76aee service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:05.400613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 901/1790] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:05.486834 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2952d-d6fd-495b-9eb1-3c1149fb6aca admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:05.504129 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06421359-729f-436e-a02a-1d2fbd1cb66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:05.505104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06421359-729f-436e-a02a-1d2fbd1cb66c admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:05.505385 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06421359-729f-436e-a02a-1d2fbd1cb66c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:05.505660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06421359-729f-436e-a02a-1d2fbd1cb66c admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:05.509662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06421359-729f-436e-a02a-1d2fbd1cb66c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:05.511716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06421359-729f-436e-a02a-1d2fbd1cb66c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:05.530062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 890/1791] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:05.533810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed11e22-8aef-4196-ac30-3b3e4fb76aee service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:05.545015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed11e22-8aef-4196-ac30-3b3e4fb76aee service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:05.566514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11087943-4a6e-4e5f-9a60-4b18decb2d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:05.571199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11087943-4a6e-4e5f-9a60-4b18decb2d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:05.571791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11087943-4a6e-4e5f-9a60-4b18decb2d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:05.572269 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11087943-4a6e-4e5f-9a60-4b18decb2d1d admin admin] PATH_INFO: `/v3/projects/e82e030b88414e95b3883f160eacc901/users/625e9c115dac44b1b790775ca0dd7211/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:05.583947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11087943-4a6e-4e5f-9a60-4b18decb2d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e82e030b88414e95b3883f160eacc901, user_id=625e9c115dac44b1b790775ca0dd7211, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:05.593875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11087943-4a6e-4e5f-9a60-4b18decb2d1d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:05.625119 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 902/1792] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:05 2022] PUT /identity/v3/projects/e82e030b88414e95b3883f160eacc901/users/625e9c115dac44b1b790775ca0dd7211/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:05.636417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06242e2-a8a5-4e97-b22f-137b15207326 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:05.637106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06242e2-a8a5-4e97-b22f-137b15207326 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:05.637550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06242e2-a8a5-4e97-b22f-137b15207326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:05.715832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-e06242e2-a8a5-4e97-b22f-137b15207326 None None] MFA Rules not processed for user `625e9c115dac44b1b790775ca0dd7211`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:05.717083 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e06242e2-a8a5-4e97-b22f-137b15207326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:05.893114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 891/1793] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:05.914151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11087943-4a6e-4e5f-9a60-4b18decb2d1d admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:05.917063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-11087943-4a6e-4e5f-9a60-4b18decb2d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:05.966565 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:06.139685 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11087943-4a6e-4e5f-9a60-4b18decb2d1d admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:06.157072 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eca885-22b2-4546-93c7-9d997857c616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:06.157796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eca885-22b2-4546-93c7-9d997857c616 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:06.158221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eca885-22b2-4546-93c7-9d997857c616 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:06.158600 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eca885-22b2-4546-93c7-9d997857c616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:06.166296 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f9eca885-22b2-4546-93c7-9d997857c616 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:06.327897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eca885-22b2-4546-93c7-9d997857c616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:06.329772 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eca885-22b2-4546-93c7-9d997857c616 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:06.367833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 903/1794] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:06.751095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92d8b1-dcea-49c8-af7a-7b17c1148966 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:06.755505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92d8b1-dcea-49c8-af7a-7b17c1148966 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:06.755505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c92d8b1-dcea-49c8-af7a-7b17c1148966 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:06.755505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 892/1795] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:43:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:43:06.775012 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae418e-cdab-4acf-84dd-c6da29f46252 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:06.775971 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae418e-cdab-4acf-84dd-c6da29f46252 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:06.776390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae418e-cdab-4acf-84dd-c6da29f46252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:06.837242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-b3ae418e-cdab-4acf-84dd-c6da29f46252 None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:06.855128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b3ae418e-cdab-4acf-84dd-c6da29f46252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:07.023463 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 904/1796] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:43:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:07.966169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92d8b1-dcea-49c8-af7a-7b17c1148966 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:07.970813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3c92d8b1-dcea-49c8-af7a-7b17c1148966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:08.014658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c92d8b1-dcea-49c8-af7a-7b17c1148966 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:08.044161 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcef44-fd2b-43eb-b2cb-1d295c83c6b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:08.048339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcef44-fd2b-43eb-b2cb-1d295c83c6b8 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:08.048915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcef44-fd2b-43eb-b2cb-1d295c83c6b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:08.049400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcef44-fd2b-43eb-b2cb-1d295c83c6b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:08.075493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcef44-fd2b-43eb-b2cb-1d295c83c6b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:08.077741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcef44-fd2b-43eb-b2cb-1d295c83c6b8 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:08.096687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 893/1797] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:08.351153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae418e-cdab-4acf-84dd-c6da29f46252 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:08.355145 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b3ae418e-cdab-4acf-84dd-c6da29f46252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:08.510421 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:08.529314 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae418e-cdab-4acf-84dd-c6da29f46252 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:08.540511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:08.541182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:08.541460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:08.541732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] PATH_INFO: `/v3/projects/743739cecc414beba6faaa16a378a9a0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:08.543553 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=743739cecc414beba6faaa16a378a9a0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:08.546338 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:08.703229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '743739cecc414beba6faaa16a378a9a0', 'request_id': 'req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:08.718456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '743739cecc414beba6faaa16a378a9a0', 'request_id': 'req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:08.734019 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '743739cecc414beba6faaa16a378a9a0', 'request_id': 'req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:08.822460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] The token cache is being invalidate because project 743739cecc414beba6faaa16a378a9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:08.823679 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:08.823679 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:08.826466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 905/1798] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:08 2022] DELETE /identity/v3/projects/743739cecc414beba6faaa16a378a9a0 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:08.831583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcef44-fd2b-43eb-b2cb-1d295c83c6b8 service nova] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:08.864585 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcef44-fd2b-43eb-b2cb-1d295c83c6b8 service nova] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:08.942062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:08.966499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa594e0d-fa45-44c7-83b8-d5e38165ed01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:08.967249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa594e0d-fa45-44c7-83b8-d5e38165ed01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:08.968039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa594e0d-fa45-44c7-83b8-d5e38165ed01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:08.968587 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa594e0d-fa45-44c7-83b8-d5e38165ed01 admin admin] PATH_INFO: `/v3/users/eec0f48003ae4674948012c8d541d282` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:08.970801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa594e0d-fa45-44c7-83b8-d5e38165ed01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eec0f48003ae4674948012c8d541d282)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:08.973908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa594e0d-fa45-44c7-83b8-d5e38165ed01 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:09.005562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:09.007267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:09.192752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d27f0-c6d3-40a4-a9a9-6c8d439c50fe admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:09.197425 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5b9e-5b9a-4c31-9cb7-9c0ef8390599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:09.198092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5b9e-5b9a-4c31-9cb7-9c0ef8390599 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:09.198364 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5b9e-5b9a-4c31-9cb7-9c0ef8390599 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:09.198626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5b9e-5b9a-4c31-9cb7-9c0ef8390599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:09.201647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ac1f5b9e-5b9a-4c31-9cb7-9c0ef8390599 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:09.250334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fa594e0d-fa45-44c7-83b8-d5e38165ed01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eec0f48003ae4674948012c8d541d282'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:09.254089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fa594e0d-fa45-44c7-83b8-d5e38165ed01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eec0f48003ae4674948012c8d541d282'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:09.342188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fa594e0d-fa45-44c7-83b8-d5e38165ed01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eec0f48003ae4674948012c8d541d282'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:09.353673 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fa594e0d-fa45-44c7-83b8-d5e38165ed01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eec0f48003ae4674948012c8d541d282'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:09.356764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fa594e0d-fa45-44c7-83b8-d5e38165ed01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eec0f48003ae4674948012c8d541d282'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:09.394947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f5b9e-5b9a-4c31-9cb7-9c0ef8390599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:09.397516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f5b9e-5b9a-4c31-9cb7-9c0ef8390599 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:09.410910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 894/1799] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:08 2022] DELETE /identity/v3/users/eec0f48003ae4674948012c8d541d282 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:09.431625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86495e52-9439-428a-9663-154cce93a0b4 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:09.432030 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86495e52-9439-428a-9663-154cce93a0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:09.434743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 906/1800] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:09.435040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86495e52-9439-428a-9663-154cce93a0b4 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:09.436621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 895/1801] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:43:09 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:43:09.539386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5b9e-5b9a-4c31-9cb7-9c0ef8390599 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:09.544692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ac1f5b9e-5b9a-4c31-9cb7-9c0ef8390599 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:09.556184 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9ce82-52eb-40c6-a566-f3b5dbaba982 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:09.556651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9ce82-52eb-40c6-a566-f3b5dbaba982 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:09.557129 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9ce82-52eb-40c6-a566-f3b5dbaba982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:09.582556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5b9e-5b9a-4c31-9cb7-9c0ef8390599 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:09.628822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-12f9ce82-52eb-40c6-a566-f3b5dbaba982 None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:09.631698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-12f9ce82-52eb-40c6-a566-f3b5dbaba982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:09.774657 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:09.821405 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 896/1802] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:43:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:09.832612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:09.833254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:09.833556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:09.833786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] PATH_INFO: `/v3/projects/c25ae3f893dd4d2eb1da0641addded20` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:09.835260 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c25ae3f893dd4d2eb1da0641addded20)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:09.837215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:09.979064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9ce82-52eb-40c6-a566-f3b5dbaba982 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:09.981415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-12f9ce82-52eb-40c6-a566-f3b5dbaba982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:10.018528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9ce82-52eb-40c6-a566-f3b5dbaba982 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:10.051913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c25ae3f893dd4d2eb1da0641addded20', 'request_id': 'req-4805fb6b-fe31-4838-a2df-088f55a66660'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:10.086989 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c25ae3f893dd4d2eb1da0641addded20', 'request_id': 'req-4805fb6b-fe31-4838-a2df-088f55a66660'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:10.104838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c25ae3f893dd4d2eb1da0641addded20', 'request_id': 'req-4805fb6b-fe31-4838-a2df-088f55a66660'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:10.168011 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:10.201958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] The token cache is being invalidate because project c25ae3f893dd4d2eb1da0641addded20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:10.202281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:10.205261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:10.213409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 907/1803] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:09 2022] DELETE /identity/v3/projects/c25ae3f893dd4d2eb1da0641addded20 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:10.216949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b353d6e3-8187-41a0-8eb0-9db66b09f486 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:10.217986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353d6e3-8187-41a0-8eb0-9db66b09f486 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:10.218502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353d6e3-8187-41a0-8eb0-9db66b09f486 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:10.219630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b353d6e3-8187-41a0-8eb0-9db66b09f486 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:10.222737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:10.224064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b353d6e3-8187-41a0-8eb0-9db66b09f486 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:10.229905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:10.249698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353d6e3-8187-41a0-8eb0-9db66b09f486 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:10.252575 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b353d6e3-8187-41a0-8eb0-9db66b09f486 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:10.268111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 897/1804] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:10.442934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805fb6b-fe31-4838-a2df-088f55a66660 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:10.449693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:10.451238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:10.451238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:10.451238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] PATH_INFO: `/v3/users/e323b30956904754ac6cc03c640c4fc0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:10.452570 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=e323b30956904754ac6cc03c640c4fc0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:10.455183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:10.698764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e323b30956904754ac6cc03c640c4fc0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:10.702046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e323b30956904754ac6cc03c640c4fc0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:10.786211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e323b30956904754ac6cc03c640c4fc0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:10.786961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e323b30956904754ac6cc03c640c4fc0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:10.798859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e323b30956904754ac6cc03c640c4fc0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:10.830427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 908/1805] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:10 2022] DELETE /identity/v3/users/e323b30956904754ac6cc03c640c4fc0 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:11.354278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a31d3d-70d9-402a-af10-c800b9eeab53 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:11.357127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a31d3d-70d9-402a-af10-c800b9eeab53 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:11.357566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a31d3d-70d9-402a-af10-c800b9eeab53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:11.479695 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-a0a31d3d-70d9-402a-af10-c800b9eeab53 None None] MFA Rules not processed for user `b29bb91e157048af8fea6eead061bd1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:11.481579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a0a31d3d-70d9-402a-af10-c800b9eeab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:11.575052 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:11.700910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 898/1806] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:11.705526 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:11.707475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:11.772142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a31d3d-70d9-402a-af10-c800b9eeab53 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:11.774023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a0a31d3d-70d9-402a-af10-c800b9eeab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:11.887842 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f652b-bfde-452a-9410-258b7a75b2ac admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:11.901027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d73f7b-e7bd-4af9-942a-0ea450ddb69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:11.903903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d73f7b-e7bd-4af9-942a-0ea450ddb69c admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:11.904441 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d73f7b-e7bd-4af9-942a-0ea450ddb69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:11.904871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d73f7b-e7bd-4af9-942a-0ea450ddb69c admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:11.905680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d73f7b-e7bd-4af9-942a-0ea450ddb69c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:11.911753 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d73f7b-e7bd-4af9-942a-0ea450ddb69c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:12.016169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a31d3d-70d9-402a-af10-c800b9eeab53 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:12.022908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14e9ea-a4ee-493b-acfe-5776e36a6318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:12.024393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14e9ea-a4ee-493b-acfe-5776e36a6318 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:12.024974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14e9ea-a4ee-493b-acfe-5776e36a6318 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:12.025775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14e9ea-a4ee-493b-acfe-5776e36a6318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:12.035068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-bc14e9ea-a4ee-493b-acfe-5776e36a6318 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:12.051289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 909/1807] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:11 2022] POST /identity/v3/projects => generated 373 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:12.058276 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d73f7b-e7bd-4af9-942a-0ea450ddb69c admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:12.250429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d73f7b-e7bd-4af9-942a-0ea450ddb69c admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:12.257642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde26461-e115-42c3-a89a-5e2a9a678141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:12.263692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde26461-e115-42c3-a89a-5e2a9a678141 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:12.264020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde26461-e115-42c3-a89a-5e2a9a678141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:12.264259 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde26461-e115-42c3-a89a-5e2a9a678141 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:12.264920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde26461-e115-42c3-a89a-5e2a9a678141 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:12.267234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde26461-e115-42c3-a89a-5e2a9a678141 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:12.267445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14e9ea-a4ee-493b-acfe-5776e36a6318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:12.268889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14e9ea-a4ee-493b-acfe-5776e36a6318 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:12.297947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 899/1808] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:43:12.318800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14e9ea-a4ee-493b-acfe-5776e36a6318 service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:12.320291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-bc14e9ea-a4ee-493b-acfe-5776e36a6318 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:12.345447 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14e9ea-a4ee-493b-acfe-5776e36a6318 service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:12.355178 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597589b-add5-416d-9441-181582d70650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:12.356052 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597589b-add5-416d-9441-181582d70650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:12.356496 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597589b-add5-416d-9441-181582d70650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:12.356890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597589b-add5-416d-9441-181582d70650 admin admin] PATH_INFO: `/v3/projects/d18b47746c9147a7aee6cbd3dd70fe45` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:12.370810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597589b-add5-416d-9441-181582d70650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d18b47746c9147a7aee6cbd3dd70fe45)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:12.372740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597589b-add5-416d-9441-181582d70650 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:12.398221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:12.444057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 910/1809] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:12 2022] POST /identity/v3/users => generated 293 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:12.446772 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde26461-e115-42c3-a89a-5e2a9a678141 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:12.495458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde26461-e115-42c3-a89a-5e2a9a678141 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:12.501107 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cab44a-6ecb-4363-b2ff-a85fdc9dee69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:12.501851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cab44a-6ecb-4363-b2ff-a85fdc9dee69 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:12.502102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cab44a-6ecb-4363-b2ff-a85fdc9dee69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:12.502381 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cab44a-6ecb-4363-b2ff-a85fdc9dee69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:12.503123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cab44a-6ecb-4363-b2ff-a85fdc9dee69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:12.505068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cab44a-6ecb-4363-b2ff-a85fdc9dee69 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:12.547086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 911/1810] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:12.560294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cab44a-6ecb-4363-b2ff-a85fdc9dee69 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:12.590292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b597589b-add5-416d-9441-181582d70650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd18b47746c9147a7aee6cbd3dd70fe45', 'request_id': 'req-b597589b-add5-416d-9441-181582d70650'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:12.590292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b597589b-add5-416d-9441-181582d70650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd18b47746c9147a7aee6cbd3dd70fe45', 'request_id': 'req-b597589b-add5-416d-9441-181582d70650'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:12.600057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cab44a-6ecb-4363-b2ff-a85fdc9dee69 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:12.606293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b597589b-add5-416d-9441-181582d70650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd18b47746c9147a7aee6cbd3dd70fe45', 'request_id': 'req-b597589b-add5-416d-9441-181582d70650'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:12.753893 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b597589b-add5-416d-9441-181582d70650 admin admin] The token cache is being invalidate because project d18b47746c9147a7aee6cbd3dd70fe45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:12.754703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b597589b-add5-416d-9441-181582d70650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:12.766351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b597589b-add5-416d-9441-181582d70650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:12.769413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 900/1811] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:12 2022] DELETE /identity/v3/projects/d18b47746c9147a7aee6cbd3dd70fe45 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:12.775553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6ebf-38a9-4d50-9e65-f2ed16376d30 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:12.776458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6ebf-38a9-4d50-9e65-f2ed16376d30 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:12.777135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b6ebf-38a9-4d50-9e65-f2ed16376d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:12.823859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0723b9c-4f1e-412c-8189-68e515792092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:12.826237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0723b9c-4f1e-412c-8189-68e515792092 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:12.826720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0723b9c-4f1e-412c-8189-68e515792092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:12.827190 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0723b9c-4f1e-412c-8189-68e515792092 admin admin] PATH_INFO: `/v3/projects/9d2e4b0153ff4563b91a906926526018/users/987b20939db94554b6226728a38f8e61/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:12.835920 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0f1b6ebf-38a9-4d50-9e65-f2ed16376d30 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:12.843200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0f1b6ebf-38a9-4d50-9e65-f2ed16376d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:12.870716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0723b9c-4f1e-412c-8189-68e515792092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d2e4b0153ff4563b91a906926526018, user_id=987b20939db94554b6226728a38f8e61, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:12.883886 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0723b9c-4f1e-412c-8189-68e515792092 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:12.893483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 901/1812] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:12.900066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b6e93-e033-464c-9b53-15d27f7b8420 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:12.900066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b6e93-e033-464c-9b53-15d27f7b8420 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:12.900066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656b6e93-e033-464c-9b53-15d27f7b8420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:12.949945 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-656b6e93-e033-464c-9b53-15d27f7b8420 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:12.951886 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-656b6e93-e033-464c-9b53-15d27f7b8420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:12.985423 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 902/1813] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:12.986426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b6e93-e033-464c-9b53-15d27f7b8420 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:13.020896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656b6e93-e033-464c-9b53-15d27f7b8420 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:13.021126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 912/1814] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:12 2022] PUT /identity/v3/projects/9d2e4b0153ff4563b91a906926526018/users/987b20939db94554b6226728a38f8e61/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:13.021971 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0723b9c-4f1e-412c-8189-68e515792092 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:13.028255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c0723b9c-4f1e-412c-8189-68e515792092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:13.184555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920326e-c1c7-4a58-8ad8-ab563359363e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:13.187010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920326e-c1c7-4a58-8ad8-ab563359363e admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:13.187010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920326e-c1c7-4a58-8ad8-ab563359363e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:13.187010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920326e-c1c7-4a58-8ad8-ab563359363e admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:13.187528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920326e-c1c7-4a58-8ad8-ab563359363e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:43:13.187826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920326e-c1c7-4a58-8ad8-ab563359363e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:13.196584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920326e-c1c7-4a58-8ad8-ab563359363e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:13.209284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0723b9c-4f1e-412c-8189-68e515792092 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:13.230731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970d1d2-9c27-419c-985d-14049051823a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:13.230731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970d1d2-9c27-419c-985d-14049051823a admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:13.230731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970d1d2-9c27-419c-985d-14049051823a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:13.230731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970d1d2-9c27-419c-985d-14049051823a admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:13.231116 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970d1d2-9c27-419c-985d-14049051823a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:13.232387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970d1d2-9c27-419c-985d-14049051823a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:13.252648 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 913/1815] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:13.252763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 903/1816] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:43:13.256935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970d1d2-9c27-419c-985d-14049051823a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:13.257106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920326e-c1c7-4a58-8ad8-ab563359363e admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:13.279222 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970d1d2-9c27-419c-985d-14049051823a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:13.289183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814c744-814c-44db-810a-5771adfa4a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:13.290010 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814c744-814c-44db-810a-5771adfa4a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:13.290443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814c744-814c-44db-810a-5771adfa4a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:13.290555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920326e-c1c7-4a58-8ad8-ab563359363e admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:13.293701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814c744-814c-44db-810a-5771adfa4a7c admin admin] PATH_INFO: `/v3/projects/9d2e4b0153ff4563b91a906926526018/users/987b20939db94554b6226728a38f8e61/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:13.293701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814c744-814c-44db-810a-5771adfa4a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d2e4b0153ff4563b91a906926526018, user_id=987b20939db94554b6226728a38f8e61, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:13.297358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814c744-814c-44db-810a-5771adfa4a7c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:13.321596 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400d08b-757d-469c-8445-78d88964f9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:13.322622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400d08b-757d-469c-8445-78d88964f9bd admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:13.324797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400d08b-757d-469c-8445-78d88964f9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:13.324921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 914/1817] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:13 2022] PUT /identity/v3/projects/9d2e4b0153ff4563b91a906926526018/users/987b20939db94554b6226728a38f8e61/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:13.325560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400d08b-757d-469c-8445-78d88964f9bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:13.326256 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400d08b-757d-469c-8445-78d88964f9bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:13.328520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400d08b-757d-469c-8445-78d88964f9bd admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:13.470854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 904/1818] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:13 2022] POST /identity/v3/projects => generated 363 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:13.475610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814c744-814c-44db-810a-5771adfa4a7c admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:13.636646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814c744-814c-44db-810a-5771adfa4a7c admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:13.756568 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400d08b-757d-469c-8445-78d88964f9bd admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:13.756973 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c400d08b-757d-469c-8445-78d88964f9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:13.779020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb9c68-97e7-423b-b77c-b49700492392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:13.780642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb9c68-97e7-423b-b77c-b49700492392 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:13.780999 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb9c68-97e7-423b-b77c-b49700492392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:13.782155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb9c68-97e7-423b-b77c-b49700492392 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:13.785417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb9c68-97e7-423b-b77c-b49700492392 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:13.791443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400d08b-757d-469c-8445-78d88964f9bd admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:13.792217 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb9c68-97e7-423b-b77c-b49700492392 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:13.813732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978aae99-3c75-4335-844f-9d7b010ce8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:13.817348 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978aae99-3c75-4335-844f-9d7b010ce8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:13.817622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978aae99-3c75-4335-844f-9d7b010ce8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:13.817918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978aae99-3c75-4335-844f-9d7b010ce8fa admin admin] PATH_INFO: `/v3/projects/c0487478220e447fb7ceef03ad440af5` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:13.822995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978aae99-3c75-4335-844f-9d7b010ce8fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0487478220e447fb7ceef03ad440af5)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:13.830426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978aae99-3c75-4335-844f-9d7b010ce8fa admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:13.895399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 915/1819] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:13 2022] POST /identity/v3/users => generated 282 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:13.902537 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb9c68-97e7-423b-b77c-b49700492392 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:13.935426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb9c68-97e7-423b-b77c-b49700492392 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:13.954892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614aefe-2df3-47ee-adbc-c03f7b9ea7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:13.956913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614aefe-2df3-47ee-adbc-c03f7b9ea7cb admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:13.956913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614aefe-2df3-47ee-adbc-c03f7b9ea7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:13.956913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614aefe-2df3-47ee-adbc-c03f7b9ea7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:13.960885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614aefe-2df3-47ee-adbc-c03f7b9ea7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:13.965997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614aefe-2df3-47ee-adbc-c03f7b9ea7cb admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:14.026769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 916/1820] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:14.026769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614aefe-2df3-47ee-adbc-c03f7b9ea7cb admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:14.069100 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:14.074399 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-978aae99-3c75-4335-844f-9d7b010ce8fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0487478220e447fb7ceef03ad440af5', 'request_id': 'req-978aae99-3c75-4335-844f-9d7b010ce8fa'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:14.076346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614aefe-2df3-47ee-adbc-c03f7b9ea7cb admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:14.106350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-978aae99-3c75-4335-844f-9d7b010ce8fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0487478220e447fb7ceef03ad440af5', 'request_id': 'req-978aae99-3c75-4335-844f-9d7b010ce8fa'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:14.139917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-978aae99-3c75-4335-844f-9d7b010ce8fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0487478220e447fb7ceef03ad440af5', 'request_id': 'req-978aae99-3c75-4335-844f-9d7b010ce8fa'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:14.231302 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b64895-5ad1-4275-b8b3-9464f9a623fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:14.232018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b64895-5ad1-4275-b8b3-9464f9a623fa admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:14.232290 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b64895-5ad1-4275-b8b3-9464f9a623fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:14.232561 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b64895-5ad1-4275-b8b3-9464f9a623fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:14.233226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b64895-5ad1-4275-b8b3-9464f9a623fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:14.234905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b64895-5ad1-4275-b8b3-9464f9a623fa admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:14.248067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 917/1821] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:14.251907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b64895-5ad1-4275-b8b3-9464f9a623fa admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:14.273188 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:14.280578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-978aae99-3c75-4335-844f-9d7b010ce8fa admin admin] The token cache is being invalidate because project c0487478220e447fb7ceef03ad440af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:14.282458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-978aae99-3c75-4335-844f-9d7b010ce8fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:14.282458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-978aae99-3c75-4335-844f-9d7b010ce8fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:14.285972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 905/1822] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:13 2022] DELETE /identity/v3/projects/c0487478220e447fb7ceef03ad440af5 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:14.286127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b64895-5ad1-4275-b8b3-9464f9a623fa admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:14.289661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2f74-57ee-47b9-9f91-47dd56b59097 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:14.290024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2f74-57ee-47b9-9f91-47dd56b59097 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:14.290241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2f74-57ee-47b9-9f91-47dd56b59097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:14.310323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175cc16-efef-424d-89fa-7f2694913f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:14.313778 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175cc16-efef-424d-89fa-7f2694913f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:14.314315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175cc16-efef-424d-89fa-7f2694913f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:14.316435 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175cc16-efef-424d-89fa-7f2694913f83 admin admin] PATH_INFO: `/v3/projects/a2aaa9bd04ae4c84adc8720c1b8499dd/users/9e01c0223dff4937be3d99b6ccb9e855/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:14.333712 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-3daf2f74-57ee-47b9-9f91-47dd56b59097 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:14.335688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175cc16-efef-424d-89fa-7f2694913f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2aaa9bd04ae4c84adc8720c1b8499dd, user_id=9e01c0223dff4937be3d99b6ccb9e855, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:14.337814 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3daf2f74-57ee-47b9-9f91-47dd56b59097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:14.338821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175cc16-efef-424d-89fa-7f2694913f83 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:14.356981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 906/1823] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:14.366296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf74669-7923-4c69-aa30-3642fd0c89df None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:14.366660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf74669-7923-4c69-aa30-3642fd0c89df None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:14.366995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf74669-7923-4c69-aa30-3642fd0c89df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:14.411846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-2cf74669-7923-4c69-aa30-3642fd0c89df None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:14.414502 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2cf74669-7923-4c69-aa30-3642fd0c89df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:14.453847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 918/1824] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:14 2022] PUT /identity/v3/projects/a2aaa9bd04ae4c84adc8720c1b8499dd/users/9e01c0223dff4937be3d99b6ccb9e855/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:14.457953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 907/1825] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:14.463025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf74669-7923-4c69-aa30-3642fd0c89df None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:14.476906 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ccdb6-8add-4d51-9709-d6a34078b525 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:14.477397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ccdb6-8add-4d51-9709-d6a34078b525 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:14.477915 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ccdb6-8add-4d51-9709-d6a34078b525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:14.479958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf74669-7923-4c69-aa30-3642fd0c89df None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:14.555693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-449ccdb6-8add-4d51-9709-d6a34078b525 None None] MFA Rules not processed for user `9e01c0223dff4937be3d99b6ccb9e855`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:14.557210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-449ccdb6-8add-4d51-9709-d6a34078b525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:14.614630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e5713-3723-4ee4-a06f-4758f018fd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:14.614630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e5713-3723-4ee4-a06f-4758f018fd51 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:14.615134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e5713-3723-4ee4-a06f-4758f018fd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:14.615419 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e5713-3723-4ee4-a06f-4758f018fd51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:14.616593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e5713-3723-4ee4-a06f-4758f018fd51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:43:14.617136 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e5713-3723-4ee4-a06f-4758f018fd51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:14.619172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e5713-3723-4ee4-a06f-4758f018fd51 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:14.645529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 908/1826] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:43:14.649949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e5713-3723-4ee4-a06f-4758f018fd51 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:14.672333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 919/1827] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:14.677808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ccdb6-8add-4d51-9709-d6a34078b525 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:14.687504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-449ccdb6-8add-4d51-9709-d6a34078b525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:14.704335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e5713-3723-4ee4-a06f-4758f018fd51 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:14.719597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafe1bd-2c87-4f19-ad21-9c8bb409b5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:14.720339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafe1bd-2c87-4f19-ad21-9c8bb409b5ea admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:14.720703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafe1bd-2c87-4f19-ad21-9c8bb409b5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:14.721356 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafe1bd-2c87-4f19-ad21-9c8bb409b5ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:14.722735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafe1bd-2c87-4f19-ad21-9c8bb409b5ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:14.727281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafe1bd-2c87-4f19-ad21-9c8bb409b5ea admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:14.732431 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ccdb6-8add-4d51-9709-d6a34078b525 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:14.738643 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e905ec-f961-442f-9feb-3b2c01d20292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:14.739997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e905ec-f961-442f-9feb-3b2c01d20292 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:14.740618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e905ec-f961-442f-9feb-3b2c01d20292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:14.741869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e905ec-f961-442f-9feb-3b2c01d20292 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:14.742667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e905ec-f961-442f-9feb-3b2c01d20292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:14.745927 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e905ec-f961-442f-9feb-3b2c01d20292 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:14.881217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 909/1828] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:14 2022] POST /identity/v3/projects => generated 365 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:14.886496 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafe1bd-2c87-4f19-ad21-9c8bb409b5ea admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:14.930688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 920/1829] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:14 2022] POST /identity/v3/projects => generated 365 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:43:14.932755 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e905ec-f961-442f-9feb-3b2c01d20292 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:15.039846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafe1bd-2c87-4f19-ad21-9c8bb409b5ea admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:15.095401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e905ec-f961-442f-9feb-3b2c01d20292 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:15.110612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8712c6-002f-4002-b91f-35197e4f873e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:15.110612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8712c6-002f-4002-b91f-35197e4f873e admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:15.110612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8712c6-002f-4002-b91f-35197e4f873e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:15.110612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8712c6-002f-4002-b91f-35197e4f873e admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:15.110889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8712c6-002f-4002-b91f-35197e4f873e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:15.110889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8712c6-002f-4002-b91f-35197e4f873e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:15.211790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8c115-0bd3-494e-88e6-c9bb668a0b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:15.212683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8c115-0bd3-494e-88e6-c9bb668a0b58 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:15.213154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8c115-0bd3-494e-88e6-c9bb668a0b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:15.213616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8c115-0bd3-494e-88e6-c9bb668a0b58 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:15.214509 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8c115-0bd3-494e-88e6-c9bb668a0b58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:15.216620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e8c115-0bd3-494e-88e6-c9bb668a0b58 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:15.274975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 921/1830] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:14 2022] POST /identity/v3/users => generated 289 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:15.279270 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8712c6-002f-4002-b91f-35197e4f873e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:15.328966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8712c6-002f-4002-b91f-35197e4f873e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:15.348117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e33fb-5272-4b86-90c1-ab6922292b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:15.348747 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e33fb-5272-4b86-90c1-ab6922292b1b admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:15.349055 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e33fb-5272-4b86-90c1-ab6922292b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:15.349339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e33fb-5272-4b86-90c1-ab6922292b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:15.350106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e33fb-5272-4b86-90c1-ab6922292b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:15.351846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e33fb-5272-4b86-90c1-ab6922292b1b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:15.368447 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 922/1831] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:15.374835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e33fb-5272-4b86-90c1-ab6922292b1b admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:15.389947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e33fb-5272-4b86-90c1-ab6922292b1b admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:15.403597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 910/1832] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:14 2022] POST /identity/v3/users => generated 283 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:15.404668 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8c115-0bd3-494e-88e6-c9bb668a0b58 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:15.434178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe73be8-2980-40dc-8489-f26ffbcffd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:15.435340 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe73be8-2980-40dc-8489-f26ffbcffd12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:15.435647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe73be8-2980-40dc-8489-f26ffbcffd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:15.435922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe73be8-2980-40dc-8489-f26ffbcffd12 admin admin] PATH_INFO: `/v3/projects/7dfc2e5a84564167ac87fe68c60c7882/users/f6222418d1364ee397180f99a17462d6/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:15.448422 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8c115-0bd3-494e-88e6-c9bb668a0b58 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:15.460404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06faa5-938c-46db-8eab-fbd00b48c638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:15.461118 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06faa5-938c-46db-8eab-fbd00b48c638 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:15.461680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06faa5-938c-46db-8eab-fbd00b48c638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:15.462000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06faa5-938c-46db-8eab-fbd00b48c638 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:15.463898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06faa5-938c-46db-8eab-fbd00b48c638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:15.468403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06faa5-938c-46db-8eab-fbd00b48c638 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:15.475492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe73be8-2980-40dc-8489-f26ffbcffd12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dfc2e5a84564167ac87fe68c60c7882, user_id=f6222418d1364ee397180f99a17462d6, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:15.478288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe73be8-2980-40dc-8489-f26ffbcffd12 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:15.483190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 911/1833] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:15.487643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06faa5-938c-46db-8eab-fbd00b48c638 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:15.508194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06faa5-938c-46db-8eab-fbd00b48c638 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:15.516011 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ac0cf-42ad-48b4-aed0-b533b01100ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:15.516998 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ac0cf-42ad-48b4-aed0-b533b01100ef admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:15.517615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ac0cf-42ad-48b4-aed0-b533b01100ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:15.518102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ac0cf-42ad-48b4-aed0-b533b01100ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:15.519772 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ac0cf-42ad-48b4-aed0-b533b01100ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:15.521862 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ac0cf-42ad-48b4-aed0-b533b01100ef admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:15.543940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 912/1834] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:43:15.548176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ac0cf-42ad-48b4-aed0-b533b01100ef admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:15.582152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 923/1835] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:15 2022] PUT /identity/v3/projects/7dfc2e5a84564167ac87fe68c60c7882/users/f6222418d1364ee397180f99a17462d6/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:15.586577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe73be8-2980-40dc-8489-f26ffbcffd12 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:15.606876 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ac0cf-42ad-48b4-aed0-b533b01100ef admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:15.711932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b7dfc-fe1e-4d01-bbd1-988dfd7bdb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:15.713965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b7dfc-fe1e-4d01-bbd1-988dfd7bdb8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:15.715232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b7dfc-fe1e-4d01-bbd1-988dfd7bdb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:15.715612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b7dfc-fe1e-4d01-bbd1-988dfd7bdb8b admin admin] PATH_INFO: `/v3/projects/d3f965d3d5974d8e8a4e998d241bb12f/users/0ae22f772674476791e4b27fa201a960/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:15.718645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe73be8-2980-40dc-8489-f26ffbcffd12 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:15.727084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf8264-ddfe-4d4c-b6ff-7f36c1464d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:15.728029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf8264-ddfe-4d4c-b6ff-7f36c1464d76 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:15.728478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf8264-ddfe-4d4c-b6ff-7f36c1464d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:15.728897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf8264-ddfe-4d4c-b6ff-7f36c1464d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:15.729687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf8264-ddfe-4d4c-b6ff-7f36c1464d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:15.731663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf8264-ddfe-4d4c-b6ff-7f36c1464d76 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:15.750352 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 924/1836] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:43:15.756235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf8264-ddfe-4d4c-b6ff-7f36c1464d76 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:15.773542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf8264-ddfe-4d4c-b6ff-7f36c1464d76 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:15.781274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e6af9-610f-40cb-9d49-6623622ef565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:15.782775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e6af9-610f-40cb-9d49-6623622ef565 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:15.783220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e6af9-610f-40cb-9d49-6623622ef565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:15.783767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e6af9-610f-40cb-9d49-6623622ef565 admin admin] PATH_INFO: `/v3/projects/7dfc2e5a84564167ac87fe68c60c7882/users/f6222418d1364ee397180f99a17462d6/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:15.787185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e6af9-610f-40cb-9d49-6623622ef565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dfc2e5a84564167ac87fe68c60c7882, user_id=f6222418d1364ee397180f99a17462d6, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:15.788499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b7dfc-fe1e-4d01-bbd1-988dfd7bdb8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f965d3d5974d8e8a4e998d241bb12f, user_id=0ae22f772674476791e4b27fa201a960, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:15.792419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e6af9-610f-40cb-9d49-6623622ef565 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:15.794830 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b7dfc-fe1e-4d01-bbd1-988dfd7bdb8b admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:15.825041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 925/1837] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:15 2022] PUT /identity/v3/projects/7dfc2e5a84564167ac87fe68c60c7882/users/f6222418d1364ee397180f99a17462d6/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:15.843962 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb773b-7752-4056-990f-8c7fa9516246 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:15.844242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb773b-7752-4056-990f-8c7fa9516246 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:15.844515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb773b-7752-4056-990f-8c7fa9516246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:15.919972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 913/1838] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:15 2022] PUT /identity/v3/projects/d3f965d3d5974d8e8a4e998d241bb12f/users/0ae22f772674476791e4b27fa201a960/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:15.921729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef675c9-fe40-4142-ae66-71b35865ad8a None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:15.922529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef675c9-fe40-4142-ae66-71b35865ad8a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:15.922529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef675c9-fe40-4142-ae66-71b35865ad8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:15.925512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-18cb773b-7752-4056-990f-8c7fa9516246 None None] MFA Rules not processed for user `f6222418d1364ee397180f99a17462d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:15.929327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-18cb773b-7752-4056-990f-8c7fa9516246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:15.988031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-3ef675c9-fe40-4142-ae66-71b35865ad8a None None] MFA Rules not processed for user `0ae22f772674476791e4b27fa201a960`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:15.991128 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3ef675c9-fe40-4142-ae66-71b35865ad8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:16.069377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 926/1839] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:16.082003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb773b-7752-4056-990f-8c7fa9516246 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:16.084299 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-18cb773b-7752-4056-990f-8c7fa9516246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:16.111263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 914/1840] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:16.112475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef675c9-fe40-4142-ae66-71b35865ad8a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:16.190975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb773b-7752-4056-990f-8c7fa9516246 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:16.195093 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6af4d-e0cc-4d89-b298-3edc60a361b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:16.196282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6af4d-e0cc-4d89-b298-3edc60a361b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:16.196282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6af4d-e0cc-4d89-b298-3edc60a361b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:16.196481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6af4d-e0cc-4d89-b298-3edc60a361b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:16.206984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6af4d-e0cc-4d89-b298-3edc60a361b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:16.208516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6af4d-e0cc-4d89-b298-3edc60a361b4 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:16.235439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 927/1841] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:43:16.238558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef675c9-fe40-4142-ae66-71b35865ad8a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:16.243720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c678b7d-6f25-4562-b6c6-c97ef6bcf48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:16.244713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c678b7d-6f25-4562-b6c6-c97ef6bcf48d admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:16.245022 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c678b7d-6f25-4562-b6c6-c97ef6bcf48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:16.245350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c678b7d-6f25-4562-b6c6-c97ef6bcf48d admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:16.246368 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c678b7d-6f25-4562-b6c6-c97ef6bcf48d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:16.248599 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c678b7d-6f25-4562-b6c6-c97ef6bcf48d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:16.397179 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:16.407055 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 915/1842] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:16 2022] POST /identity/v3/projects => generated 365 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:43:16.411746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6af4d-e0cc-4d89-b298-3edc60a361b4 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:16.432291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6af4d-e0cc-4d89-b298-3edc60a361b4 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:16.477490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:16.518080 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c3aac-cbb8-47d1-9766-a2a8266d5cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:16.519524 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c3aac-cbb8-47d1-9766-a2a8266d5cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:16.519940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c3aac-cbb8-47d1-9766-a2a8266d5cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:16.520311 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c3aac-cbb8-47d1-9766-a2a8266d5cb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:16.521115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c3aac-cbb8-47d1-9766-a2a8266d5cb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:16.523310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c3aac-cbb8-47d1-9766-a2a8266d5cb5 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:16.648143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 928/1843] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:16 2022] POST /identity/v3/users => generated 289 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:43:16.652106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c678b7d-6f25-4562-b6c6-c97ef6bcf48d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:16.662635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c678b7d-6f25-4562-b6c6-c97ef6bcf48d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:16.674389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a443e-288a-43bb-a2b8-4dad3bbe43d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:16.677816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a443e-288a-43bb-a2b8-4dad3bbe43d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:16.677816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a443e-288a-43bb-a2b8-4dad3bbe43d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:16.677816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a443e-288a-43bb-a2b8-4dad3bbe43d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:16.678657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a443e-288a-43bb-a2b8-4dad3bbe43d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:16.682705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a443e-288a-43bb-a2b8-4dad3bbe43d7 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:16.707756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 916/1844] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:16.709165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c3aac-cbb8-47d1-9766-a2a8266d5cb5 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:16.720755 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c3aac-cbb8-47d1-9766-a2a8266d5cb5 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:16.730246 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361a638-64bc-4050-8b52-82cb889c2807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:16.731040 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361a638-64bc-4050-8b52-82cb889c2807 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:16.731456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361a638-64bc-4050-8b52-82cb889c2807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:16.731817 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5361a638-64bc-4050-8b52-82cb889c2807 admin admin] PATH_INFO: `/v3/projects/c4e7765628f842b194c26177dc40d32a/users/36206dcab9a04205a62f9a9a0a8ae444/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:16.762929 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361a638-64bc-4050-8b52-82cb889c2807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e7765628f842b194c26177dc40d32a, user_id=36206dcab9a04205a62f9a9a0a8ae444, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:16.768400 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5361a638-64bc-4050-8b52-82cb889c2807 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:16.888444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 929/1845] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:16 2022] PUT /identity/v3/projects/c4e7765628f842b194c26177dc40d32a/users/36206dcab9a04205a62f9a9a0a8ae444/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:16.892550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a443e-288a-43bb-a2b8-4dad3bbe43d7 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:16.902407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a443e-288a-43bb-a2b8-4dad3bbe43d7 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:16.987147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeeb487-75d6-481f-b2b9-20025fe0a6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:16.991513 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeeb487-75d6-481f-b2b9-20025fe0a6aa admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:16.991819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeeb487-75d6-481f-b2b9-20025fe0a6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:16.992097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeeb487-75d6-481f-b2b9-20025fe0a6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:16.992883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeeb487-75d6-481f-b2b9-20025fe0a6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:16.994475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeeb487-75d6-481f-b2b9-20025fe0a6aa admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:17.014518 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 917/1846] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:16 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:17.016477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361a638-64bc-4050-8b52-82cb889c2807 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:17.038185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5361a638-64bc-4050-8b52-82cb889c2807 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:17.046741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee4f1b-33f4-433a-a76f-2fcbb0f5d3c3 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:17.046986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee4f1b-33f4-433a-a76f-2fcbb0f5d3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:17.047263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee4f1b-33f4-433a-a76f-2fcbb0f5d3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:17.057643 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f6075-a87b-41c3-9959-fb9acd3aafc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:17.064320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f6075-a87b-41c3-9959-fb9acd3aafc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:17.064320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f6075-a87b-41c3-9959-fb9acd3aafc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:17.064320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f6075-a87b-41c3-9959-fb9acd3aafc0 admin admin] PATH_INFO: `/v3/projects/c4e7765628f842b194c26177dc40d32a/users/36206dcab9a04205a62f9a9a0a8ae444/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:17.064320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f6075-a87b-41c3-9959-fb9acd3aafc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e7765628f842b194c26177dc40d32a, user_id=36206dcab9a04205a62f9a9a0a8ae444, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:17.065964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f6075-a87b-41c3-9959-fb9acd3aafc0 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:17.084744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 930/1847] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:17 2022] PUT /identity/v3/projects/c4e7765628f842b194c26177dc40d32a/users/36206dcab9a04205a62f9a9a0a8ae444/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:17.099219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adec598-639e-407b-b8ee-db3a8c1f98d1 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:17.099835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adec598-639e-407b-b8ee-db3a8c1f98d1 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:17.100275 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adec598-639e-407b-b8ee-db3a8c1f98d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:17.114692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-66ee4f1b-33f4-433a-a76f-2fcbb0f5d3c3 None None] MFA Rules not processed for user `987b20939db94554b6226728a38f8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:17.117251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-66ee4f1b-33f4-433a-a76f-2fcbb0f5d3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:17.170102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-6adec598-639e-407b-b8ee-db3a8c1f98d1 None None] MFA Rules not processed for user `36206dcab9a04205a62f9a9a0a8ae444`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:17.173237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-6adec598-639e-407b-b8ee-db3a8c1f98d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:17.239857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 918/1848] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:17.254843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee4f1b-33f4-433a-a76f-2fcbb0f5d3c3 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:17.256305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-66ee4f1b-33f4-433a-a76f-2fcbb0f5d3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:17.269868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 931/1849] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:17.315450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ee4f1b-33f4-433a-a76f-2fcbb0f5d3c3 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:17.319097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c3334-78ce-4cca-b5f0-94d6e5e64727 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:17.319692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c3334-78ce-4cca-b5f0-94d6e5e64727 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:17.319964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c3334-78ce-4cca-b5f0-94d6e5e64727 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:17.320227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c3334-78ce-4cca-b5f0-94d6e5e64727 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:17.322132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-506c3334-78ce-4cca-b5f0-94d6e5e64727 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:17.323567 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:17.360442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adec598-639e-407b-b8ee-db3a8c1f98d1 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:17.372258 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adec598-639e-407b-b8ee-db3a8c1f98d1 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:17.376146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c5715-e5d8-408d-8312-9a14efe57c54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:17.376870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c5715-e5d8-408d-8312-9a14efe57c54 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:17.377204 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c5715-e5d8-408d-8312-9a14efe57c54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:17.377469 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588c5715-e5d8-408d-8312-9a14efe57c54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:17.389126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c5715-e5d8-408d-8312-9a14efe57c54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:17.389332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c3334-78ce-4cca-b5f0-94d6e5e64727 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:17.390648 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588c5715-e5d8-408d-8312-9a14efe57c54 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:17.390928 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c3334-78ce-4cca-b5f0-94d6e5e64727 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:17.401263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 919/1850] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:17.402071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 932/1851] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:17.579655 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c3334-78ce-4cca-b5f0-94d6e5e64727 service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:17.664293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c3334-78ce-4cca-b5f0-94d6e5e64727 service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:17.670368 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:17.671010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:17.671288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:17.672382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] PATH_INFO: `/v3/users/9e01c0223dff4937be3d99b6ccb9e855` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:17.672678 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e01c0223dff4937be3d99b6ccb9e855)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:17.674809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:17.717554 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c5715-e5d8-408d-8312-9a14efe57c54 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:17.725956 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588c5715-e5d8-408d-8312-9a14efe57c54 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:17.730298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b934b1-0da3-459e-9c2a-8ed9cabab101 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:17.731171 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b934b1-0da3-459e-9c2a-8ed9cabab101 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:17.731606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b934b1-0da3-459e-9c2a-8ed9cabab101 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:17.731876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b934b1-0da3-459e-9c2a-8ed9cabab101 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:17.774963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:17.793141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b934b1-0da3-459e-9c2a-8ed9cabab101 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:17.794602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b934b1-0da3-459e-9c2a-8ed9cabab101 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:17.804304 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 933/1852] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:17.869079 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e01c0223dff4937be3d99b6ccb9e855'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:17.869726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e01c0223dff4937be3d99b6ccb9e855'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:17.946315 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e01c0223dff4937be3d99b6ccb9e855'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:17.954063 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e01c0223dff4937be3d99b6ccb9e855'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:17.954725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e01c0223dff4937be3d99b6ccb9e855'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:17.974570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 920/1853] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:17 2022] DELETE /identity/v3/users/9e01c0223dff4937be3d99b6ccb9e855 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:17.979115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff41dd-0f3b-4f60-bfe1-be57b8b28686 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:17.979415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff41dd-0f3b-4f60-bfe1-be57b8b28686 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:17.984650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff41dd-0f3b-4f60-bfe1-be57b8b28686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:17.999794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-97ff41dd-0f3b-4f60-bfe1-be57b8b28686 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:18.000976 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-97ff41dd-0f3b-4f60-bfe1-be57b8b28686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:18.074308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 934/1854] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:18.081927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:18.083049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:18.147252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7bbfe4-bdd7-496f-af95-7e46310481cc admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:18.150853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759e46b-6102-4f38-96b9-6b5d09b6a6c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:18.151461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759e46b-6102-4f38-96b9-6b5d09b6a6c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:18.151739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759e46b-6102-4f38-96b9-6b5d09b6a6c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:18.152001 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759e46b-6102-4f38-96b9-6b5d09b6a6c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:18.154465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff41dd-0f3b-4f60-bfe1-be57b8b28686 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:18.156156 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-97ff41dd-0f3b-4f60-bfe1-be57b8b28686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:18.162477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759e46b-6102-4f38-96b9-6b5d09b6a6c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:18.163897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759e46b-6102-4f38-96b9-6b5d09b6a6c9 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:18.169620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff41dd-0f3b-4f60-bfe1-be57b8b28686 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:18.173549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 921/1855] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:18.174533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:18.175703 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:18.176289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:18.176511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] PATH_INFO: `/v3/users/4b19d6c7585e42e59300a9ba395f8759` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:18.177669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b19d6c7585e42e59300a9ba395f8759)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:18.179722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:18.470837 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b19d6c7585e42e59300a9ba395f8759'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:18.471693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b19d6c7585e42e59300a9ba395f8759'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:18.529064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b19d6c7585e42e59300a9ba395f8759'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:18.529737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b19d6c7585e42e59300a9ba395f8759'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:18.540262 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b19d6c7585e42e59300a9ba395f8759'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:18.566773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 935/1856] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:18 2022] DELETE /identity/v3/users/4b19d6c7585e42e59300a9ba395f8759 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:18.571508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefec74-88f6-4c3e-a7c4-dc0cf686a2a1 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:18.571819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefec74-88f6-4c3e-a7c4-dc0cf686a2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:18.572094 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cefec74-88f6-4c3e-a7c4-dc0cf686a2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:18.594277 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-3cefec74-88f6-4c3e-a7c4-dc0cf686a2a1 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:18.594277 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3cefec74-88f6-4c3e-a7c4-dc0cf686a2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:18.655991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 922/1857] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:18.660430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:18.665784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:18.667195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:18.723448 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b251d3e-7f9f-494d-b7da-1a45572ee246 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:18.727097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b616a9-0177-4a72-b008-abda19192a21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:18.727707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b616a9-0177-4a72-b008-abda19192a21 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:18.727983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b616a9-0177-4a72-b008-abda19192a21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:18.728253 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b616a9-0177-4a72-b008-abda19192a21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:18.738130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b616a9-0177-4a72-b008-abda19192a21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:18.739554 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b616a9-0177-4a72-b008-abda19192a21 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:18.752524 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 936/1858] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:18.754493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefec74-88f6-4c3e-a7c4-dc0cf686a2a1 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:18.755958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3cefec74-88f6-4c3e-a7c4-dc0cf686a2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:18.768915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cefec74-88f6-4c3e-a7c4-dc0cf686a2a1 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:18.774260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:18.775229 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:18.775657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:18.775935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] PATH_INFO: `/v3/projects/a2aaa9bd04ae4c84adc8720c1b8499dd` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:18.777579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2aaa9bd04ae4c84adc8720c1b8499dd)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:18.779441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:18.947616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2aaa9bd04ae4c84adc8720c1b8499dd', 'request_id': 'req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:18.957187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2aaa9bd04ae4c84adc8720c1b8499dd', 'request_id': 'req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:18.965334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2aaa9bd04ae4c84adc8720c1b8499dd', 'request_id': 'req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:19.049199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] The token cache is being invalidate because project a2aaa9bd04ae4c84adc8720c1b8499dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:19.049587 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:19.054698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:19.054698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 923/1859] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:18 2022] DELETE /identity/v3/projects/a2aaa9bd04ae4c84adc8720c1b8499dd => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:19.057650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b616a9-0177-4a72-b008-abda19192a21 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:19.068891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b616a9-0177-4a72-b008-abda19192a21 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:19.151980 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:19.157634 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:19.157634 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:19.157634 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] PATH_INFO: `/v3/users/f6222418d1364ee397180f99a17462d6` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:19.157634 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6222418d1364ee397180f99a17462d6)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:19.157634 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:19.194535 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:19.196211 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:19.255662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd14209-ce8b-4ba0-a8a2-3c114117ea83 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:19.259579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd7673-bb6e-42d8-8dc6-8281fa3e9e8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:19.260304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd7673-bb6e-42d8-8dc6-8281fa3e9e8c service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:19.261632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:19.262100 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd7673-bb6e-42d8-8dc6-8281fa3e9e8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:19.262423 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd7673-bb6e-42d8-8dc6-8281fa3e9e8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:19.264150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c7cd7673-bb6e-42d8-8dc6-8281fa3e9e8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:19.328249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd7673-bb6e-42d8-8dc6-8281fa3e9e8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:19.330359 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd7673-bb6e-42d8-8dc6-8281fa3e9e8c service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:19.344644 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 924/1860] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:43:19.347374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd7673-bb6e-42d8-8dc6-8281fa3e9e8c service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:19.348627 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c7cd7673-bb6e-42d8-8dc6-8281fa3e9e8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:19.377661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6222418d1364ee397180f99a17462d6'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:19.379840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd7673-bb6e-42d8-8dc6-8281fa3e9e8c service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:19.380690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6222418d1364ee397180f99a17462d6'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:19.389314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:19.390059 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:19.390404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:19.390791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] PATH_INFO: `/v3/projects/e6243987d41741bfb728405997d57536` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:19.392000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6243987d41741bfb728405997d57536)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:19.395407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:19.443251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6222418d1364ee397180f99a17462d6'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:19.443961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6222418d1364ee397180f99a17462d6'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:19.452306 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6222418d1364ee397180f99a17462d6'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:19.485872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 937/1861] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:19 2022] DELETE /identity/v3/users/f6222418d1364ee397180f99a17462d6 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:19.552968 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6243987d41741bfb728405997d57536', 'request_id': 'req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:19.566295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6243987d41741bfb728405997d57536', 'request_id': 'req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:19.587040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6243987d41741bfb728405997d57536', 'request_id': 'req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:19.661409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] The token cache is being invalidate because project e6243987d41741bfb728405997d57536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:19.662042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:19.662936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:19.665010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 925/1862] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:19 2022] DELETE /identity/v3/projects/e6243987d41741bfb728405997d57536 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:19.668984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:19.670536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:19.756068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431a0fc-353a-42a9-9743-4be9c4cf7629 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:19.763486 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:19.861195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381d39c-1d13-45d1-98c2-f7f778337287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:19.864923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381d39c-1d13-45d1-98c2-f7f778337287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:19.865455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381d39c-1d13-45d1-98c2-f7f778337287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:19.865947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381d39c-1d13-45d1-98c2-f7f778337287 admin admin] PATH_INFO: `/v3/users/fb2778bf13f94f92bbb34071d826126d` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:19.868345 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381d39c-1d13-45d1-98c2-f7f778337287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb2778bf13f94f92bbb34071d826126d)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:19.871790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381d39c-1d13-45d1-98c2-f7f778337287 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:20.065878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:20.067428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:20.089583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2cdc5f-94a6-4781-8ed9-ed8c30b9162e admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:20.102612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ee4e2-3d52-414e-883a-160d492dc3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:20.103265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ee4e2-3d52-414e-883a-160d492dc3f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:20.103541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ee4e2-3d52-414e-883a-160d492dc3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:20.103541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ee4e2-3d52-414e-883a-160d492dc3f5 admin admin] PATH_INFO: `/v3/projects/7dfc2e5a84564167ac87fe68c60c7882` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:20.104996 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ee4e2-3d52-414e-883a-160d492dc3f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dfc2e5a84564167ac87fe68c60c7882)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:20.106988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ee4e2-3d52-414e-883a-160d492dc3f5 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:20.161901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8381d39c-1d13-45d1-98c2-f7f778337287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb2778bf13f94f92bbb34071d826126d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:20.162834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8381d39c-1d13-45d1-98c2-f7f778337287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb2778bf13f94f92bbb34071d826126d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:20.241040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:20.255181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8381d39c-1d13-45d1-98c2-f7f778337287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb2778bf13f94f92bbb34071d826126d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:20.255821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8381d39c-1d13-45d1-98c2-f7f778337287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb2778bf13f94f92bbb34071d826126d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:20.264805 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8381d39c-1d13-45d1-98c2-f7f778337287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb2778bf13f94f92bbb34071d826126d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:20.274660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cb7ee4e2-3d52-414e-883a-160d492dc3f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dfc2e5a84564167ac87fe68c60c7882', 'request_id': 'req-cb7ee4e2-3d52-414e-883a-160d492dc3f5'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:20.281177 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cb7ee4e2-3d52-414e-883a-160d492dc3f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dfc2e5a84564167ac87fe68c60c7882', 'request_id': 'req-cb7ee4e2-3d52-414e-883a-160d492dc3f5'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:20.286488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 938/1863] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:19 2022] DELETE /identity/v3/users/fb2778bf13f94f92bbb34071d826126d => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:20.294819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cb7ee4e2-3d52-414e-883a-160d492dc3f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dfc2e5a84564167ac87fe68c60c7882', 'request_id': 'req-cb7ee4e2-3d52-414e-883a-160d492dc3f5'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:20.391401 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cb7ee4e2-3d52-414e-883a-160d492dc3f5 admin admin] The token cache is being invalidate because project 7dfc2e5a84564167ac87fe68c60c7882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:20.391691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cb7ee4e2-3d52-414e-883a-160d492dc3f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:20.392342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cb7ee4e2-3d52-414e-883a-160d492dc3f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:20.395905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 926/1864] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:20 2022] DELETE /identity/v3/projects/7dfc2e5a84564167ac87fe68c60c7882 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:20.399476 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ea06c-7742-41ce-a3e8-3a65ee490ef3 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:20.399759 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ea06c-7742-41ce-a3e8-3a65ee490ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:20.400049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ea06c-7742-41ce-a3e8-3a65ee490ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:20.420231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-6c5ea06c-7742-41ce-a3e8-3a65ee490ef3 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:20.421422 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-6c5ea06c-7742-41ce-a3e8-3a65ee490ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:20.478897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 939/1865] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:20.486881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c0cdd-3d0d-4c21-8b7f-88f5e2ea250e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:20.487240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c0cdd-3d0d-4c21-8b7f-88f5e2ea250e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:20.487920 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c0cdd-3d0d-4c21-8b7f-88f5e2ea250e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:20.511054 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-4b4c0cdd-3d0d-4c21-8b7f-88f5e2ea250e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:20.511939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4b4c0cdd-3d0d-4c21-8b7f-88f5e2ea250e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:20.527700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 927/1866] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:20.529790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ea06c-7742-41ce-a3e8-3a65ee490ef3 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:20.541005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5ea06c-7742-41ce-a3e8-3a65ee490ef3 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:20.546450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9468574-2425-4a63-b116-22a2f411d313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:20.547277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9468574-2425-4a63-b116-22a2f411d313 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:20.547698 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9468574-2425-4a63-b116-22a2f411d313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:20.548122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9468574-2425-4a63-b116-22a2f411d313 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:20.548957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9468574-2425-4a63-b116-22a2f411d313 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:43:20.549399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9468574-2425-4a63-b116-22a2f411d313 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:20.551214 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9468574-2425-4a63-b116-22a2f411d313 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:20.565950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 940/1867] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:43:20.569948 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c0cdd-3d0d-4c21-8b7f-88f5e2ea250e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:20.583883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c0cdd-3d0d-4c21-8b7f-88f5e2ea250e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:20.590605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1510f4-6331-4074-a973-10d1eb206805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:20.591356 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1510f4-6331-4074-a973-10d1eb206805 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:20.591741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1510f4-6331-4074-a973-10d1eb206805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:20.592253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1510f4-6331-4074-a973-10d1eb206805 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:20.593164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1510f4-6331-4074-a973-10d1eb206805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:20.595180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1510f4-6331-4074-a973-10d1eb206805 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:20.637761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:20.673897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 928/1868] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:20 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:20.679083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9468574-2425-4a63-b116-22a2f411d313 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:20.691305 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9468574-2425-4a63-b116-22a2f411d313 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:20.751236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f39b48-4417-4c50-95b5-22699f389450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:20.751845 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f39b48-4417-4c50-95b5-22699f389450 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:20.752465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f39b48-4417-4c50-95b5-22699f389450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:20.753275 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f39b48-4417-4c50-95b5-22699f389450 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:20.753948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f39b48-4417-4c50-95b5-22699f389450 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:20.756057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f39b48-4417-4c50-95b5-22699f389450 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:20.844250 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 941/1869] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:20 2022] POST /identity/v3/users => generated 284 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:20.851040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1510f4-6331-4074-a973-10d1eb206805 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:20.866816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1510f4-6331-4074-a973-10d1eb206805 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:20.876707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf4b1d-7c19-4714-8582-05cb087b2feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:20.878153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf4b1d-7c19-4714-8582-05cb087b2feb admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:20.878536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf4b1d-7c19-4714-8582-05cb087b2feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:20.879789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf4b1d-7c19-4714-8582-05cb087b2feb admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:20.880629 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf4b1d-7c19-4714-8582-05cb087b2feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:20.883294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf4b1d-7c19-4714-8582-05cb087b2feb admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:20.897454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 929/1870] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:20.901930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f39b48-4417-4c50-95b5-22699f389450 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:20.913507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f39b48-4417-4c50-95b5-22699f389450 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:20.921144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0b77c-b5f3-4758-aac0-4f045263ee7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:20.921681 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0b77c-b5f3-4758-aac0-4f045263ee7e admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:20.922097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0b77c-b5f3-4758-aac0-4f045263ee7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:20.922358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0b77c-b5f3-4758-aac0-4f045263ee7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:20.923009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0b77c-b5f3-4758-aac0-4f045263ee7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:20.924628 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0b77c-b5f3-4758-aac0-4f045263ee7e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:20.937739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 942/1871] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:20.942702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf4b1d-7c19-4714-8582-05cb087b2feb admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:20.958831 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf4b1d-7c19-4714-8582-05cb087b2feb admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:20.966426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39829c9f-7e31-4d64-b86c-70157c954fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:20.967462 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39829c9f-7e31-4d64-b86c-70157c954fce admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:20.967872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39829c9f-7e31-4d64-b86c-70157c954fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:20.968150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39829c9f-7e31-4d64-b86c-70157c954fce admin admin] PATH_INFO: `/v3/projects/9c0440e55429438188a9aabe77cad7fc/users/d4f57009e69e417fa07601a15d921890/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:21.006389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39829c9f-7e31-4d64-b86c-70157c954fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c0440e55429438188a9aabe77cad7fc, user_id=d4f57009e69e417fa07601a15d921890, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:21.009482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39829c9f-7e31-4d64-b86c-70157c954fce admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:21.046871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0b77c-b5f3-4758-aac0-4f045263ee7e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:21.048843 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a9b0b77c-b5f3-4758-aac0-4f045263ee7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:21.072430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0b77c-b5f3-4758-aac0-4f045263ee7e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:21.079265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:21.080184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:21.081936 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:21.081936 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc admin admin] PATH_INFO: `/v3/projects/de0792ab075a4239ad92ff4d454d7673` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:21.082840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=de0792ab075a4239ad92ff4d454d7673)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:21.085005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:21.108755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 930/1872] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:20 2022] PUT /identity/v3/projects/9c0440e55429438188a9aabe77cad7fc/users/d4f57009e69e417fa07601a15d921890/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:21.127034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9099fd-99a0-477b-90ff-9fe44e89e276 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:21.127034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9099fd-99a0-477b-90ff-9fe44e89e276 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:21.127034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9099fd-99a0-477b-90ff-9fe44e89e276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:21.162582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-8a9099fd-99a0-477b-90ff-9fe44e89e276 None None] MFA Rules not processed for user `d4f57009e69e417fa07601a15d921890`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:21.163798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8a9099fd-99a0-477b-90ff-9fe44e89e276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:21.259706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 931/1873] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:21.267669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de0792ab075a4239ad92ff4d454d7673', 'request_id': 'req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:21.268609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9099fd-99a0-477b-90ff-9fe44e89e276 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:21.281957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de0792ab075a4239ad92ff4d454d7673', 'request_id': 'req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:21.304851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de0792ab075a4239ad92ff4d454d7673', 'request_id': 'req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:21.428236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc admin admin] The token cache is being invalidate because project de0792ab075a4239ad92ff4d454d7673 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:21.428554 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:21.431359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e54c89da-9920-4ee7-a5ce-09749ac3f0bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:21.434219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 943/1874] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:21 2022] DELETE /identity/v3/projects/de0792ab075a4239ad92ff4d454d7673 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:21.435790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff8314-e86a-48c6-bf97-ac1949eb8175 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:21.436073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff8314-e86a-48c6-bf97-ac1949eb8175 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:21.438671 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff8314-e86a-48c6-bf97-ac1949eb8175 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:21.441555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9099fd-99a0-477b-90ff-9fe44e89e276 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:21.443207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 944/1875] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:43:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:43:21.445483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff8314-e86a-48c6-bf97-ac1949eb8175 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:21.450020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-92ff8314-e86a-48c6-bf97-ac1949eb8175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:21.556021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27753e-02f3-493a-816c-9b3b7aae30ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:21.560361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27753e-02f3-493a-816c-9b3b7aae30ec admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:21.560361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27753e-02f3-493a-816c-9b3b7aae30ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:21.560361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27753e-02f3-493a-816c-9b3b7aae30ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:21.560361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27753e-02f3-493a-816c-9b3b7aae30ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:21.564915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27753e-02f3-493a-816c-9b3b7aae30ec admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:21.616729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff8314-e86a-48c6-bf97-ac1949eb8175 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:21.621388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:21.622059 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:21.622335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:21.622604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] PATH_INFO: `/v3/users/625e9c115dac44b1b790775ca0dd7211` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:21.623825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=625e9c115dac44b1b790775ca0dd7211)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:21.625747 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:21.735986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 932/1876] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:21 2022] POST /identity/v3/projects => generated 369 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:21.737883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe78d2b-dc4a-49d4-954d-71ec26036f5e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:21.738238 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe78d2b-dc4a-49d4-954d-71ec26036f5e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:21.738507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe78d2b-dc4a-49d4-954d-71ec26036f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:21.762041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-fbe78d2b-dc4a-49d4-954d-71ec26036f5e None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:21.763271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-fbe78d2b-dc4a-49d4-954d-71ec26036f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:21.845399 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 933/1877] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:43:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:21.846344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe78d2b-dc4a-49d4-954d-71ec26036f5e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:21.847742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-fbe78d2b-dc4a-49d4-954d-71ec26036f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:21.903880 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:21.910313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '625e9c115dac44b1b790775ca0dd7211'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:21.910313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '625e9c115dac44b1b790775ca0dd7211'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:21.946444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe78d2b-dc4a-49d4-954d-71ec26036f5e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:21.953467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cabbb-2dbb-4d2b-8a31-f42c62f5ed58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:21.954452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cabbb-2dbb-4d2b-8a31-f42c62f5ed58 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:21.954938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cabbb-2dbb-4d2b-8a31-f42c62f5ed58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:21.955456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804cabbb-2dbb-4d2b-8a31-f42c62f5ed58 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:21.957163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cabbb-2dbb-4d2b-8a31-f42c62f5ed58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:21.962905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804cabbb-2dbb-4d2b-8a31-f42c62f5ed58 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:22.020260 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '625e9c115dac44b1b790775ca0dd7211'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:22.021011 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '625e9c115dac44b1b790775ca0dd7211'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:22.028914 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '625e9c115dac44b1b790775ca0dd7211'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:22.052749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 945/1878] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:21 2022] DELETE /identity/v3/users/625e9c115dac44b1b790775ca0dd7211 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:22.053426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:22.055356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:22.060628 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 934/1879] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:21 2022] POST /identity/v3/users => generated 291 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:22.061691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cabbb-2dbb-4d2b-8a31-f42c62f5ed58 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:22.063440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-804cabbb-2dbb-4d2b-8a31-f42c62f5ed58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:22.082201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804cabbb-2dbb-4d2b-8a31-f42c62f5ed58 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:22.153602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215f903-8db0-4e04-b65d-132386f2c7b9 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:22.161994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092dde9-1824-4a2d-900f-ca8a48b8a60b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:22.162607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092dde9-1824-4a2d-900f-ca8a48b8a60b service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:22.162885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092dde9-1824-4a2d-900f-ca8a48b8a60b service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:22.163065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1f96d-869c-449c-8fce-2788bc9136a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:22.163560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092dde9-1824-4a2d-900f-ca8a48b8a60b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:22.163672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1f96d-869c-449c-8fce-2788bc9136a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:22.163953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1f96d-869c-449c-8fce-2788bc9136a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:22.164223 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1f96d-869c-449c-8fce-2788bc9136a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:22.164870 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1f96d-869c-449c-8fce-2788bc9136a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:22.165058 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f092dde9-1824-4a2d-900f-ca8a48b8a60b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:22.166501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1f96d-869c-449c-8fce-2788bc9136a1 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:22.180586 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092dde9-1824-4a2d-900f-ca8a48b8a60b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:22.182394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092dde9-1824-4a2d-900f-ca8a48b8a60b service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:22.190202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 935/1880] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:22.191792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1f96d-869c-449c-8fce-2788bc9136a1 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:22.199973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 946/1881] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:22.204565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1f96d-869c-449c-8fce-2788bc9136a1 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:22.214859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86817d61-e48f-41f0-8963-1533a098f22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:22.215810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86817d61-e48f-41f0-8963-1533a098f22e admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:22.216125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86817d61-e48f-41f0-8963-1533a098f22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:22.216799 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86817d61-e48f-41f0-8963-1533a098f22e admin admin] PATH_INFO: `/v3/projects/42f01d6bce224916b342a8173ff1ef7e/users/fcd647bcff404382a93cad5ad47da86d/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:22.241192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86817d61-e48f-41f0-8963-1533a098f22e admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f01d6bce224916b342a8173ff1ef7e, user_id=fcd647bcff404382a93cad5ad47da86d, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:22.245853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86817d61-e48f-41f0-8963-1533a098f22e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:22.316102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 936/1882] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:22 2022] PUT /identity/v3/projects/42f01d6bce224916b342a8173ff1ef7e/users/fcd647bcff404382a93cad5ad47da86d/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:22.320034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092dde9-1824-4a2d-900f-ca8a48b8a60b service nova] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:22.342288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092dde9-1824-4a2d-900f-ca8a48b8a60b service nova] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:22.443899 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b20d8-1ac7-4c44-abcb-0ecd21253cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:22.452954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b20d8-1ac7-4c44-abcb-0ecd21253cfe admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:22.452954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b20d8-1ac7-4c44-abcb-0ecd21253cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:22.452954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b20d8-1ac7-4c44-abcb-0ecd21253cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:22.452954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b20d8-1ac7-4c44-abcb-0ecd21253cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:22.452954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b20d8-1ac7-4c44-abcb-0ecd21253cfe admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:22.460245 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 947/1883] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:22.466375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86817d61-e48f-41f0-8963-1533a098f22e admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:22.482435 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86817d61-e48f-41f0-8963-1533a098f22e admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:22.488139 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b45f8-bfed-4ca7-abd6-f4ef1d98d6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:22.489033 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b45f8-bfed-4ca7-abd6-f4ef1d98d6b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:22.489388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b45f8-bfed-4ca7-abd6-f4ef1d98d6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:22.489705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b45f8-bfed-4ca7-abd6-f4ef1d98d6b7 admin admin] PATH_INFO: `/v3/projects/42f01d6bce224916b342a8173ff1ef7e/users/fcd647bcff404382a93cad5ad47da86d/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:22.491806 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b45f8-bfed-4ca7-abd6-f4ef1d98d6b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42f01d6bce224916b342a8173ff1ef7e, user_id=fcd647bcff404382a93cad5ad47da86d, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:22.494497 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3b45f8-bfed-4ca7-abd6-f4ef1d98d6b7 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:22.510143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 937/1884] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:22 2022] PUT /identity/v3/projects/42f01d6bce224916b342a8173ff1ef7e/users/fcd647bcff404382a93cad5ad47da86d/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:22.522794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa62ec-005d-4e8f-9b3c-4241b5c21ca0 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:22.523076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa62ec-005d-4e8f-9b3c-4241b5c21ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:22.523347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa62ec-005d-4e8f-9b3c-4241b5c21ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:22.552179 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-1caa62ec-005d-4e8f-9b3c-4241b5c21ca0 None None] MFA Rules not processed for user `fcd647bcff404382a93cad5ad47da86d`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:22.553372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-1caa62ec-005d-4e8f-9b3c-4241b5c21ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:22.607326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b45f8-bfed-4ca7-abd6-f4ef1d98d6b7 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:22.609083 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-ba3b45f8-bfed-4ca7-abd6-f4ef1d98d6b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:22.628814 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:22.710229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 948/1885] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:22.722142 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa62ec-005d-4e8f-9b3c-4241b5c21ca0 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:22.725253 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-1caa62ec-005d-4e8f-9b3c-4241b5c21ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:22.794436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b45f8-bfed-4ca7-abd6-f4ef1d98d6b7 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:22.800615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7776e94-44e3-4671-80ec-24c01f5aa483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:22.801585 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7776e94-44e3-4671-80ec-24c01f5aa483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:22.802534 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7776e94-44e3-4671-80ec-24c01f5aa483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:22.802996 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7776e94-44e3-4671-80ec-24c01f5aa483 admin admin] PATH_INFO: `/v3/projects/e82e030b88414e95b3883f160eacc901` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:22.804261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7776e94-44e3-4671-80ec-24c01f5aa483 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e82e030b88414e95b3883f160eacc901)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:22.806997 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7776e94-44e3-4671-80ec-24c01f5aa483 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:22.857844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caa62ec-005d-4e8f-9b3c-4241b5c21ca0 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:22.864662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a18bd8-ff39-40ec-b958-30182509cd07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:22.865350 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a18bd8-ff39-40ec-b958-30182509cd07 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:22.865682 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a18bd8-ff39-40ec-b958-30182509cd07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:22.876350 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a18bd8-ff39-40ec-b958-30182509cd07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:22.883237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a18bd8-ff39-40ec-b958-30182509cd07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:22.884976 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a18bd8-ff39-40ec-b958-30182509cd07 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:22.900959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 949/1886] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:22.997358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-f7776e94-44e3-4671-80ec-24c01f5aa483 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e82e030b88414e95b3883f160eacc901', 'request_id': 'req-f7776e94-44e3-4671-80ec-24c01f5aa483'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:23.004105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-f7776e94-44e3-4671-80ec-24c01f5aa483 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e82e030b88414e95b3883f160eacc901', 'request_id': 'req-f7776e94-44e3-4671-80ec-24c01f5aa483'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:23.011078 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-f7776e94-44e3-4671-80ec-24c01f5aa483 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e82e030b88414e95b3883f160eacc901', 'request_id': 'req-f7776e94-44e3-4671-80ec-24c01f5aa483'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:23.133815 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-f7776e94-44e3-4671-80ec-24c01f5aa483 admin admin] The token cache is being invalidate because project e82e030b88414e95b3883f160eacc901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:23.140516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-f7776e94-44e3-4671-80ec-24c01f5aa483 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:23.140516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-f7776e94-44e3-4671-80ec-24c01f5aa483 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:23.145781 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 938/1887] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:22 2022] DELETE /identity/v3/projects/e82e030b88414e95b3883f160eacc901 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:23.155415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7acec-9bb0-4b55-bfab-acb5496fc47b None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:23.155993 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7acec-9bb0-4b55-bfab-acb5496fc47b None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:23.155993 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7acec-9bb0-4b55-bfab-acb5496fc47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:23.210288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-3ea7acec-9bb0-4b55-bfab-acb5496fc47b None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:23.211511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3ea7acec-9bb0-4b55-bfab-acb5496fc47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:23.277132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 950/1888] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:23.285706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363005-2dd9-404d-b997-10e770b402f6 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:23.286220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363005-2dd9-404d-b997-10e770b402f6 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:23.286310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6363005-2dd9-404d-b997-10e770b402f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:23.307161 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-b6363005-2dd9-404d-b997-10e770b402f6 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:23.308388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b6363005-2dd9-404d-b997-10e770b402f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:23.318407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 939/1889] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:23.322798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7acec-9bb0-4b55-bfab-acb5496fc47b None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:23.339870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7acec-9bb0-4b55-bfab-acb5496fc47b None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:23.345356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e2d08-fd4d-4c49-9243-2b54030ba4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:23.345947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e2d08-fd4d-4c49-9243-2b54030ba4da admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:23.346227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e2d08-fd4d-4c49-9243-2b54030ba4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:23.347174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e2d08-fd4d-4c49-9243-2b54030ba4da admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:23.348355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e2d08-fd4d-4c49-9243-2b54030ba4da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:43:23.348680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e2d08-fd4d-4c49-9243-2b54030ba4da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:23.350345 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e2d08-fd4d-4c49-9243-2b54030ba4da admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:23.362215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 951/1890] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:43:23.366710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6363005-2dd9-404d-b997-10e770b402f6 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:23.375724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6363005-2dd9-404d-b997-10e770b402f6 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:23.380994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd9991-9ce3-4a99-ade3-55fe07a64d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:23.381615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd9991-9ce3-4a99-ade3-55fe07a64d76 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:23.381933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd9991-9ce3-4a99-ade3-55fe07a64d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:23.382197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd9991-9ce3-4a99-ade3-55fe07a64d76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:23.382817 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd9991-9ce3-4a99-ade3-55fe07a64d76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:23.384725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd9991-9ce3-4a99-ade3-55fe07a64d76 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:23.496932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 940/1891] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:23 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:23.500757 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e2d08-fd4d-4c49-9243-2b54030ba4da admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:23.526732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e2d08-fd4d-4c49-9243-2b54030ba4da admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:23.540669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:23.586060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a3b8a-ff11-485d-9380-f6e3c4f1097d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:23.586815 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a3b8a-ff11-485d-9380-f6e3c4f1097d admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:23.587066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a3b8a-ff11-485d-9380-f6e3c4f1097d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:23.587338 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a3b8a-ff11-485d-9380-f6e3c4f1097d admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:23.587965 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a3b8a-ff11-485d-9380-f6e3c4f1097d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:23.590010 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a3b8a-ff11-485d-9380-f6e3c4f1097d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:23.690867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 952/1892] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:23 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:43:23.695376 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd9991-9ce3-4a99-ade3-55fe07a64d76 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:23.706479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd9991-9ce3-4a99-ade3-55fe07a64d76 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:23.711091 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb6e83-c10d-4d8b-928b-2ed818342ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:23.712414 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb6e83-c10d-4d8b-928b-2ed818342ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:23.712414 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb6e83-c10d-4d8b-928b-2ed818342ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:23.712414 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb6e83-c10d-4d8b-928b-2ed818342ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:23.712946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb6e83-c10d-4d8b-928b-2ed818342ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:23.714551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb6e83-c10d-4d8b-928b-2ed818342ed9 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:23.725076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 941/1893] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:23.729246 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a3b8a-ff11-485d-9380-f6e3c4f1097d admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:23.739946 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a3b8a-ff11-485d-9380-f6e3c4f1097d admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:23.744525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452a0af-58b7-45de-bd20-34a711a14127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:23.745172 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452a0af-58b7-45de-bd20-34a711a14127 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:23.745448 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452a0af-58b7-45de-bd20-34a711a14127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:23.745718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452a0af-58b7-45de-bd20-34a711a14127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:23.746387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452a0af-58b7-45de-bd20-34a711a14127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:23.748072 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452a0af-58b7-45de-bd20-34a711a14127 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:23.763689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 953/1894] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:23.769485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb6e83-c10d-4d8b-928b-2ed818342ed9 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:23.780651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb6e83-c10d-4d8b-928b-2ed818342ed9 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:23.787104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190e299-7af1-4fca-9e69-e9ea0d5c6e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:23.787768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190e299-7af1-4fca-9e69-e9ea0d5c6e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:23.788049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190e299-7af1-4fca-9e69-e9ea0d5c6e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:23.788318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190e299-7af1-4fca-9e69-e9ea0d5c6e4d admin admin] PATH_INFO: `/v3/projects/c1115ed4067b40f5a751fa75fc7fc295/users/68a9a2df341b4c01b458f16de90ae88b/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:23.804208 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190e299-7af1-4fca-9e69-e9ea0d5c6e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1115ed4067b40f5a751fa75fc7fc295, user_id=68a9a2df341b4c01b458f16de90ae88b, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:23.806943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190e299-7af1-4fca-9e69-e9ea0d5c6e4d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:23.888924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 942/1895] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:23 2022] PUT /identity/v3/projects/c1115ed4067b40f5a751fa75fc7fc295/users/68a9a2df341b4c01b458f16de90ae88b/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:23.900352 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74466654-7a5a-4afc-b65f-8bf166fcb733 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:23.900777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74466654-7a5a-4afc-b65f-8bf166fcb733 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:23.901133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74466654-7a5a-4afc-b65f-8bf166fcb733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:23.956471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-74466654-7a5a-4afc-b65f-8bf166fcb733 None None] MFA Rules not processed for user `68a9a2df341b4c01b458f16de90ae88b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:23.958713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-74466654-7a5a-4afc-b65f-8bf166fcb733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:24.057390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 954/1896] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:24.062777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:24.072078 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190e299-7af1-4fca-9e69-e9ea0d5c6e4d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:24.073886 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5190e299-7af1-4fca-9e69-e9ea0d5c6e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:24.170561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190e299-7af1-4fca-9e69-e9ea0d5c6e4d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:24.174311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3302e58-f066-477b-825a-e61af6616bcd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d96ec6235843c5b67d87a3d0ff0f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:24.175555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3302e58-f066-477b-825a-e61af6616bcd service glance] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:24.175824 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3302e58-f066-477b-825a-e61af6616bcd service glance] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:24.176085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3302e58-f066-477b-825a-e61af6616bcd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:24.185059 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3302e58-f066-477b-825a-e61af6616bcd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:24.187462 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3302e58-f066-477b-825a-e61af6616bcd service glance] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:24.197306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 943/1897] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:24.609157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:25.084199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:25.683686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:26.179057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:26.380613 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:26.406650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74466654-7a5a-4afc-b65f-8bf166fcb733 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:26.410589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-74466654-7a5a-4afc-b65f-8bf166fcb733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:26.500411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74466654-7a5a-4afc-b65f-8bf166fcb733 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:26.524411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:26.525814 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:26.526360 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:26.526841 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] PATH_INFO: `/v3/users/d4f57009e69e417fa07601a15d921890` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:26.528506 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4f57009e69e417fa07601a15d921890)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:26.530992 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:26.794875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4f57009e69e417fa07601a15d921890'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:26.802036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4f57009e69e417fa07601a15d921890'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:26.849161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:26.888439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4f57009e69e417fa07601a15d921890'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:26.889212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4f57009e69e417fa07601a15d921890'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:26.899579 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4f57009e69e417fa07601a15d921890'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:26.928686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 955/1898] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:26 2022] DELETE /identity/v3/users/d4f57009e69e417fa07601a15d921890 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:26.933508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba95b6d-a6ef-49d6-965a-444904887f2f None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:26.934143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba95b6d-a6ef-49d6-965a-444904887f2f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:26.934240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba95b6d-a6ef-49d6-965a-444904887f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:26.962721 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-6ba95b6d-a6ef-49d6-965a-444904887f2f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:26.963964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6ba95b6d-a6ef-49d6-965a-444904887f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:27.023116 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 944/1899] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:27.034458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:27.035592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:27.114134 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70307b-a347-4da9-bd92-094fbe1369c4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:27.119393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:27.120287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497a127-ffc0-4b52-9efe-6d7697cb5a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:27.120974 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497a127-ffc0-4b52-9efe-6d7697cb5a86 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:27.121259 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497a127-ffc0-4b52-9efe-6d7697cb5a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:27.121526 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f497a127-ffc0-4b52-9efe-6d7697cb5a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:27.132777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497a127-ffc0-4b52-9efe-6d7697cb5a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:27.134251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f497a127-ffc0-4b52-9efe-6d7697cb5a86 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:27.154120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 956/1900] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:27.615695 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba95b6d-a6ef-49d6-965a-444904887f2f None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:27.619714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6ba95b6d-a6ef-49d6-965a-444904887f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:27.662235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba95b6d-a6ef-49d6-965a-444904887f2f None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:27.680896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:27.682041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:27.684011 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:27.684011 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] PATH_INFO: `/v3/projects/9c0440e55429438188a9aabe77cad7fc` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:27.685056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c0440e55429438188a9aabe77cad7fc)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:27.687662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:27.719885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:27.835717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c0440e55429438188a9aabe77cad7fc', 'request_id': 'req-fc3539ce-ef81-44de-83ff-c8853333a7ba'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:27.841336 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c0440e55429438188a9aabe77cad7fc', 'request_id': 'req-fc3539ce-ef81-44de-83ff-c8853333a7ba'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:27.850810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c0440e55429438188a9aabe77cad7fc', 'request_id': 'req-fc3539ce-ef81-44de-83ff-c8853333a7ba'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:27.962588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] The token cache is being invalidate because project 9c0440e55429438188a9aabe77cad7fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:27.963250 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:27.963539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:27.965044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 945/1901] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:27 2022] DELETE /identity/v3/projects/9c0440e55429438188a9aabe77cad7fc => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:27.969089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497a127-ffc0-4b52-9efe-6d7697cb5a86 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:27.980788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f497a127-ffc0-4b52-9efe-6d7697cb5a86 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:28.046246 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:28.058872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137b579-da84-4265-a6a8-721dc8e46aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:28.059644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137b579-da84-4265-a6a8-721dc8e46aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:28.059756 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137b579-da84-4265-a6a8-721dc8e46aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:28.060120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137b579-da84-4265-a6a8-721dc8e46aba admin admin] PATH_INFO: `/v3/users/fcd647bcff404382a93cad5ad47da86d` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:28.061290 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137b579-da84-4265-a6a8-721dc8e46aba admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcd647bcff404382a93cad5ad47da86d)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:28.063408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137b579-da84-4265-a6a8-721dc8e46aba admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:28.329907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5137b579-da84-4265-a6a8-721dc8e46aba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcd647bcff404382a93cad5ad47da86d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:28.332045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5137b579-da84-4265-a6a8-721dc8e46aba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcd647bcff404382a93cad5ad47da86d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:28.396961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5137b579-da84-4265-a6a8-721dc8e46aba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcd647bcff404382a93cad5ad47da86d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:28.397795 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5137b579-da84-4265-a6a8-721dc8e46aba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcd647bcff404382a93cad5ad47da86d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:28.406562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5137b579-da84-4265-a6a8-721dc8e46aba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcd647bcff404382a93cad5ad47da86d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:28.427857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 957/1902] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:27 2022] DELETE /identity/v3/users/fcd647bcff404382a93cad5ad47da86d => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:28.896750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:28.900603 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:28.965151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3539ce-ef81-44de-83ff-c8853333a7ba admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:29.061461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f8380-29de-46ac-a077-74a951acac21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:29.062170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f8380-29de-46ac-a077-74a951acac21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:29.062445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f8380-29de-46ac-a077-74a951acac21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:29.062714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f8380-29de-46ac-a077-74a951acac21 admin admin] PATH_INFO: `/v3/projects/42f01d6bce224916b342a8173ff1ef7e` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:29.064422 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f8380-29de-46ac-a077-74a951acac21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42f01d6bce224916b342a8173ff1ef7e)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:29.067065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f8380-29de-46ac-a077-74a951acac21 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:29.180312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e72f8380-29de-46ac-a077-74a951acac21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42f01d6bce224916b342a8173ff1ef7e', 'request_id': 'req-e72f8380-29de-46ac-a077-74a951acac21'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:29.186064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e72f8380-29de-46ac-a077-74a951acac21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42f01d6bce224916b342a8173ff1ef7e', 'request_id': 'req-e72f8380-29de-46ac-a077-74a951acac21'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:29.192257 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e72f8380-29de-46ac-a077-74a951acac21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42f01d6bce224916b342a8173ff1ef7e', 'request_id': 'req-e72f8380-29de-46ac-a077-74a951acac21'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:29.299555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e72f8380-29de-46ac-a077-74a951acac21 admin admin] The token cache is being invalidate because project 42f01d6bce224916b342a8173ff1ef7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:29.299831 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e72f8380-29de-46ac-a077-74a951acac21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:29.300402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e72f8380-29de-46ac-a077-74a951acac21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:29.301950 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 946/1903] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:28 2022] DELETE /identity/v3/projects/42f01d6bce224916b342a8173ff1ef7e => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:29.306954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe5c1f-050e-4c72-a833-2caa2aca63e0 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:29.311035 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe5c1f-050e-4c72-a833-2caa2aca63e0 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:29.311035 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe5c1f-050e-4c72-a833-2caa2aca63e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:29.330292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-dffe5c1f-050e-4c72-a833-2caa2aca63e0 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:29.333082 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-dffe5c1f-050e-4c72-a833-2caa2aca63e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:29.400598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 958/1904] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:29.419117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1aad64-1f50-4076-a9a1-333f8caf9175 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:29.420572 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1aad64-1f50-4076-a9a1-333f8caf9175 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:29.423829 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1aad64-1f50-4076-a9a1-333f8caf9175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:29.462862 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-9f1aad64-1f50-4076-a9a1-333f8caf9175 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:29.464380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9f1aad64-1f50-4076-a9a1-333f8caf9175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:29.478078 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 947/1905] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:29.486586 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe5c1f-050e-4c72-a833-2caa2aca63e0 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:29.496722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe5c1f-050e-4c72-a833-2caa2aca63e0 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:29.503144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a687d3-b21c-4bf0-9bfe-a4b72932126e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:29.503858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a687d3-b21c-4bf0-9bfe-a4b72932126e admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:29.504169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a687d3-b21c-4bf0-9bfe-a4b72932126e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:29.504437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a687d3-b21c-4bf0-9bfe-a4b72932126e admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:29.505104 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a687d3-b21c-4bf0-9bfe-a4b72932126e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:43:29.505381 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a687d3-b21c-4bf0-9bfe-a4b72932126e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:29.507087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a687d3-b21c-4bf0-9bfe-a4b72932126e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:29.521316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 959/1906] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:43:29.525978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1aad64-1f50-4076-a9a1-333f8caf9175 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:29.540622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1aad64-1f50-4076-a9a1-333f8caf9175 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:29.546522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7dbc5-8aa1-4c92-9f1d-4cccb5825997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:29.547821 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7dbc5-8aa1-4c92-9f1d-4cccb5825997 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:29.548295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7dbc5-8aa1-4c92-9f1d-4cccb5825997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:29.548774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7dbc5-8aa1-4c92-9f1d-4cccb5825997 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:29.549733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7dbc5-8aa1-4c92-9f1d-4cccb5825997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:29.552736 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7dbc5-8aa1-4c92-9f1d-4cccb5825997 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:29.675249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 948/1907] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:29 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:29.686500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a687d3-b21c-4bf0-9bfe-a4b72932126e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:29.703846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a687d3-b21c-4bf0-9bfe-a4b72932126e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:29.781589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:29.788894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057819b-1e75-4d29-a46d-d54ea31d9b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:29.790350 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057819b-1e75-4d29-a46d-d54ea31d9b1a admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:29.790627 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057819b-1e75-4d29-a46d-d54ea31d9b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:29.790891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057819b-1e75-4d29-a46d-d54ea31d9b1a admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:29.791541 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057819b-1e75-4d29-a46d-d54ea31d9b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:29.793466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057819b-1e75-4d29-a46d-d54ea31d9b1a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:29.884806 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 960/1908] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:29 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:29.889011 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7dbc5-8aa1-4c92-9f1d-4cccb5825997 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:29.900064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7dbc5-8aa1-4c92-9f1d-4cccb5825997 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:29.908237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ce738-5507-4fd7-a0cb-eaa78bf17cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:29.909776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ce738-5507-4fd7-a0cb-eaa78bf17cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:29.910145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ce738-5507-4fd7-a0cb-eaa78bf17cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:29.911267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ce738-5507-4fd7-a0cb-eaa78bf17cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:29.911371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ce738-5507-4fd7-a0cb-eaa78bf17cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:29.915571 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ce738-5507-4fd7-a0cb-eaa78bf17cf9 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:29.917621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:29.937632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 949/1909] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:29.938711 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057819b-1e75-4d29-a46d-d54ea31d9b1a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:29.957285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057819b-1e75-4d29-a46d-d54ea31d9b1a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:29.968095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e395176-693f-4e51-8890-99d61f19bde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:29.968907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e395176-693f-4e51-8890-99d61f19bde1 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:29.969454 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e395176-693f-4e51-8890-99d61f19bde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:29.969717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e395176-693f-4e51-8890-99d61f19bde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:29.970402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e395176-693f-4e51-8890-99d61f19bde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:29.972934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e395176-693f-4e51-8890-99d61f19bde1 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:29.984377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 961/1910] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:43:29.988573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ce738-5507-4fd7-a0cb-eaa78bf17cf9 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:30.000725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ce738-5507-4fd7-a0cb-eaa78bf17cf9 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:30.005457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb54db-b3bc-47b3-bba8-69e49f9bcc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:30.006177 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb54db-b3bc-47b3-bba8-69e49f9bcc3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:30.006454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb54db-b3bc-47b3-bba8-69e49f9bcc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:30.006723 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb54db-b3bc-47b3-bba8-69e49f9bcc3c admin admin] PATH_INFO: `/v3/projects/55bc48fbb48c4fec8d68edc565be6add/users/596ecbd94d344675b708c6e4b7350eea/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:30.023850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb54db-b3bc-47b3-bba8-69e49f9bcc3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=55bc48fbb48c4fec8d68edc565be6add, user_id=596ecbd94d344675b708c6e4b7350eea, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:30.026564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb54db-b3bc-47b3-bba8-69e49f9bcc3c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:30.106041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 950/1911] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:29 2022] PUT /identity/v3/projects/55bc48fbb48c4fec8d68edc565be6add/users/596ecbd94d344675b708c6e4b7350eea/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:30.119636 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d710db1-965e-4620-a06f-d59d583cef62 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:30.119933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d710db1-965e-4620-a06f-d59d583cef62 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:30.120227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d710db1-965e-4620-a06f-d59d583cef62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:30.162846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-5d710db1-965e-4620-a06f-d59d583cef62 None None] MFA Rules not processed for user `596ecbd94d344675b708c6e4b7350eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:30.164337 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5d710db1-965e-4620-a06f-d59d583cef62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:30.276509 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 962/1912] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:30.281041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb54db-b3bc-47b3-bba8-69e49f9bcc3c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:30.422414 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb54db-b3bc-47b3-bba8-69e49f9bcc3c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:30.432397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757f72e-a586-49ee-9e46-33bbf16e6129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:30.433012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757f72e-a586-49ee-9e46-33bbf16e6129 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:30.433286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757f72e-a586-49ee-9e46-33bbf16e6129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:30.433551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757f72e-a586-49ee-9e46-33bbf16e6129 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:30.434211 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757f72e-a586-49ee-9e46-33bbf16e6129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:30.439111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757f72e-a586-49ee-9e46-33bbf16e6129 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:31.276081 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 951/1913] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:30 2022] POST /identity/v3/projects => generated 387 bytes in 996 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:43:31.280320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d710db1-965e-4620-a06f-d59d583cef62 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:31.296852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d710db1-965e-4620-a06f-d59d583cef62 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:31.360253 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ab0df-8c13-4c2e-9625-77fb2455db8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:31.360861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ab0df-8c13-4c2e-9625-77fb2455db8d admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:31.361125 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ab0df-8c13-4c2e-9625-77fb2455db8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:31.361721 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ab0df-8c13-4c2e-9625-77fb2455db8d admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:31.364067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ab0df-8c13-4c2e-9625-77fb2455db8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:31.369099 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ab0df-8c13-4c2e-9625-77fb2455db8d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:32.047832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757f72e-a586-49ee-9e46-33bbf16e6129 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:32.052451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 963/1914] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:31 2022] POST /identity/v3/users => generated 300 bytes in 767 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:43:32.066641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757f72e-a586-49ee-9e46-33bbf16e6129 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:32.083796 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cac8c5-2fa3-4700-bea7-299dd6993041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:32.084636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cac8c5-2fa3-4700-bea7-299dd6993041 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:32.085118 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cac8c5-2fa3-4700-bea7-299dd6993041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:32.085568 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cac8c5-2fa3-4700-bea7-299dd6993041 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:32.086565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cac8c5-2fa3-4700-bea7-299dd6993041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:32.088694 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cac8c5-2fa3-4700-bea7-299dd6993041 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:32.109379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 952/1915] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:43:32.115312 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ab0df-8c13-4c2e-9625-77fb2455db8d admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:32.130294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ab0df-8c13-4c2e-9625-77fb2455db8d admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:32.134945 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a650be-c1ff-405b-893d-6058e9e7260c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:32.135612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a650be-c1ff-405b-893d-6058e9e7260c admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:32.135891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a650be-c1ff-405b-893d-6058e9e7260c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:32.136161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a650be-c1ff-405b-893d-6058e9e7260c admin admin] PATH_INFO: `/v3/projects/8b7907bc40c44973b8073a37eb039489/users/618b9df44cd2487bba1dbaa868291285/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:32.166108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a650be-c1ff-405b-893d-6058e9e7260c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7907bc40c44973b8073a37eb039489, user_id=618b9df44cd2487bba1dbaa868291285, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:32.169840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a650be-c1ff-405b-893d-6058e9e7260c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:32.305864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 964/1916] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:32 2022] PUT /identity/v3/projects/8b7907bc40c44973b8073a37eb039489/users/618b9df44cd2487bba1dbaa868291285/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:32.308577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cac8c5-2fa3-4700-bea7-299dd6993041 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:32.327663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cac8c5-2fa3-4700-bea7-299dd6993041 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:32.426367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20458983-bed6-4abc-9382-c235dea04d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:32.427736 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20458983-bed6-4abc-9382-c235dea04d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:32.428042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20458983-bed6-4abc-9382-c235dea04d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:32.428205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20458983-bed6-4abc-9382-c235dea04d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:32.428892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20458983-bed6-4abc-9382-c235dea04d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:32.431181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20458983-bed6-4abc-9382-c235dea04d41 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:32.445197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 953/1917] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:32.449301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a650be-c1ff-405b-893d-6058e9e7260c admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:32.461759 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a650be-c1ff-405b-893d-6058e9e7260c admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:32.467227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d32fa-393f-4f5d-a73d-36b5371aac7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:32.468160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d32fa-393f-4f5d-a73d-36b5371aac7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:32.468596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d32fa-393f-4f5d-a73d-36b5371aac7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:32.468945 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d32fa-393f-4f5d-a73d-36b5371aac7c admin admin] PATH_INFO: `/v3/projects/8b7907bc40c44973b8073a37eb039489/users/618b9df44cd2487bba1dbaa868291285/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:32.471304 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d32fa-393f-4f5d-a73d-36b5371aac7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b7907bc40c44973b8073a37eb039489, user_id=618b9df44cd2487bba1dbaa868291285, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:32.474883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d32fa-393f-4f5d-a73d-36b5371aac7c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:32.486001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 965/1918] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:32 2022] PUT /identity/v3/projects/8b7907bc40c44973b8073a37eb039489/users/618b9df44cd2487bba1dbaa868291285/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:32.497546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2def3-e763-4acd-a47a-e54529c004e5 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:32.497855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2def3-e763-4acd-a47a-e54529c004e5 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:32.498140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2def3-e763-4acd-a47a-e54529c004e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:32.532846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-ffb2def3-e763-4acd-a47a-e54529c004e5 None None] MFA Rules not processed for user `618b9df44cd2487bba1dbaa868291285`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:32.534778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-ffb2def3-e763-4acd-a47a-e54529c004e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:32.567242 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:32.618929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 954/1919] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:32.632539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d32fa-393f-4f5d-a73d-36b5371aac7c admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:32.634412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-1f7d32fa-393f-4f5d-a73d-36b5371aac7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:32.657526 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:32.695809 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d32fa-393f-4f5d-a73d-36b5371aac7c admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:32.699701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3efc0-0b04-4ebb-b85b-a7e6c9137eba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:32.702877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3efc0-0b04-4ebb-b85b-a7e6c9137eba service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:32.702877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3efc0-0b04-4ebb-b85b-a7e6c9137eba service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:32.702877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3efc0-0b04-4ebb-b85b-a7e6c9137eba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:32.712278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3efc0-0b04-4ebb-b85b-a7e6c9137eba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:32.713802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3efc0-0b04-4ebb-b85b-a7e6c9137eba service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:32.727680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 966/1920] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:32.880288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2def3-e763-4acd-a47a-e54529c004e5 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:32.950023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2def3-e763-4acd-a47a-e54529c004e5 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:32.954487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:32.955153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:32.955430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:32.955821 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] PATH_INFO: `/v3/users/596ecbd94d344675b708c6e4b7350eea` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:32.956965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=596ecbd94d344675b708c6e4b7350eea)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:32.958955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:33.113064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '596ecbd94d344675b708c6e4b7350eea'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:33.113781 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '596ecbd94d344675b708c6e4b7350eea'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:33.178872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '596ecbd94d344675b708c6e4b7350eea'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:33.186331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '596ecbd94d344675b708c6e4b7350eea'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:33.187042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '596ecbd94d344675b708c6e4b7350eea'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:33.209374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 955/1921] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:32 2022] DELETE /identity/v3/users/596ecbd94d344675b708c6e4b7350eea => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:33.211504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6f90f-4bd1-439a-bf57-7a68c365f554 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:33.211783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6f90f-4bd1-439a-bf57-7a68c365f554 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:33.212052 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6f90f-4bd1-439a-bf57-7a68c365f554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:33.231958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-d1e6f90f-4bd1-439a-bf57-7a68c365f554 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:33.233196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d1e6f90f-4bd1-439a-bf57-7a68c365f554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:33.293058 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 967/1922] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:33.303340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:33.308860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:33.368706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:33.373962 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe6fc8-3ac3-4fde-821a-47caca2ab350 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:33.378565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c4bc8-8a5c-48f5-a338-d6a5790b8de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:33.390815 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c4bc8-8a5c-48f5-a338-d6a5790b8de9 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:33.390815 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c4bc8-8a5c-48f5-a338-d6a5790b8de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:33.390815 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c4bc8-8a5c-48f5-a338-d6a5790b8de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:33.390815 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c4bc8-8a5c-48f5-a338-d6a5790b8de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:33.390815 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c4bc8-8a5c-48f5-a338-d6a5790b8de9 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:33.401025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 956/1923] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:43:33.635473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6f90f-4bd1-439a-bf57-7a68c365f554 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:33.638810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d1e6f90f-4bd1-439a-bf57-7a68c365f554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:33.666556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6f90f-4bd1-439a-bf57-7a68c365f554 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:33.673135 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f075f1a-3720-4e02-be6d-905d59f66a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:33.673816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f075f1a-3720-4e02-be6d-905d59f66a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:33.674132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f075f1a-3720-4e02-be6d-905d59f66a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:33.680289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f075f1a-3720-4e02-be6d-905d59f66a7d admin admin] PATH_INFO: `/v3/users/0ae22f772674476791e4b27fa201a960` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:33.680289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f075f1a-3720-4e02-be6d-905d59f66a7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ae22f772674476791e4b27fa201a960)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:33.680289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f075f1a-3720-4e02-be6d-905d59f66a7d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:33.894195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1f075f1a-3720-4e02-be6d-905d59f66a7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ae22f772674476791e4b27fa201a960'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:33.895961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1f075f1a-3720-4e02-be6d-905d59f66a7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ae22f772674476791e4b27fa201a960'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.025613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1f075f1a-3720-4e02-be6d-905d59f66a7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ae22f772674476791e4b27fa201a960'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.026236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1f075f1a-3720-4e02-be6d-905d59f66a7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ae22f772674476791e4b27fa201a960'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.033511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c4bc8-8a5c-48f5-a338-d6a5790b8de9 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:34.036278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9c3c4bc8-8a5c-48f5-a338-d6a5790b8de9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:34.040298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1f075f1a-3720-4e02-be6d-905d59f66a7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ae22f772674476791e4b27fa201a960'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.060013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c4bc8-8a5c-48f5-a338-d6a5790b8de9 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:34.072480 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:34.073131 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:34.073404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:34.073848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] PATH_INFO: `/v3/projects/55bc48fbb48c4fec8d68edc565be6add` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:34.075046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] RBAC: Authorizing `identity:delete_project(project_id=55bc48fbb48c4fec8d68edc565be6add)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:34.077021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:34.079210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 968/1924] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:33 2022] DELETE /identity/v3/users/0ae22f772674476791e4b27fa201a960 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:34.088008 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da0585-82b0-4de7-bf6a-bba24a1cd2e1 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:34.088402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da0585-82b0-4de7-bf6a-bba24a1cd2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:34.088593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da0585-82b0-4de7-bf6a-bba24a1cd2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:34.113695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-88da0585-82b0-4de7-bf6a-bba24a1cd2e1 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:34.114957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-88da0585-82b0-4de7-bf6a-bba24a1cd2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:34.145750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:34.182585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 969/1925] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:34.189702 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da0585-82b0-4de7-bf6a-bba24a1cd2e1 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:34.190792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55bc48fbb48c4fec8d68edc565be6add', 'request_id': 'req-94ea294f-bcef-4d79-8477-2114773fb0de'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.192591 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-88da0585-82b0-4de7-bf6a-bba24a1cd2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:34.198930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55bc48fbb48c4fec8d68edc565be6add', 'request_id': 'req-94ea294f-bcef-4d79-8477-2114773fb0de'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.207421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55bc48fbb48c4fec8d68edc565be6add', 'request_id': 'req-94ea294f-bcef-4d79-8477-2114773fb0de'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.250023 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da0585-82b0-4de7-bf6a-bba24a1cd2e1 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:34.301156 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94dd042-e361-40e5-a4ad-7510a02b9263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:34.301769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dd042-e361-40e5-a4ad-7510a02b9263 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:34.302106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dd042-e361-40e5-a4ad-7510a02b9263 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:34.302381 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94dd042-e361-40e5-a4ad-7510a02b9263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:34.334583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] The token cache is being invalidate because project 55bc48fbb48c4fec8d68edc565be6add was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:34.335126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.335729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.337335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 957/1926] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:34 2022] DELETE /identity/v3/projects/55bc48fbb48c4fec8d68edc565be6add => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:34.341472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:34.342972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:34.367593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94dd042-e361-40e5-a4ad-7510a02b9263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:34.369086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94dd042-e361-40e5-a4ad-7510a02b9263 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:34.378936 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 970/1927] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:34.415593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea294f-bcef-4d79-8477-2114773fb0de admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:34.420498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:34.421205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:34.421482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:34.421749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] PATH_INFO: `/v3/users/618b9df44cd2487bba1dbaa868291285` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:34.422933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] RBAC: Authorizing `identity:delete_user(user_id=618b9df44cd2487bba1dbaa868291285)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:34.426274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:34.612971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '618b9df44cd2487bba1dbaa868291285'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.613753 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '618b9df44cd2487bba1dbaa868291285'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.671304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '618b9df44cd2487bba1dbaa868291285'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.680020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '618b9df44cd2487bba1dbaa868291285'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.680729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '618b9df44cd2487bba1dbaa868291285'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:34.689180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:34.699980 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 958/1928] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:34 2022] DELETE /identity/v3/users/618b9df44cd2487bba1dbaa868291285 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:34.839603 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94dd042-e361-40e5-a4ad-7510a02b9263 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:34.841624 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a94dd042-e361-40e5-a4ad-7510a02b9263 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:34.858330 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94dd042-e361-40e5-a4ad-7510a02b9263 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:34.913794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:34.943126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:34.943939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:34.944289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:34.944649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] PATH_INFO: `/v3/projects/d3f965d3d5974d8e8a4e998d241bb12f` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:34.945794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3f965d3d5974d8e8a4e998d241bb12f)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:34.947815 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:35.079786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f965d3d5974d8e8a4e998d241bb12f', 'request_id': 'req-adaf33e3-1c9e-470a-badb-61c7995d9cec'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.102300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f965d3d5974d8e8a4e998d241bb12f', 'request_id': 'req-adaf33e3-1c9e-470a-badb-61c7995d9cec'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.126645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f965d3d5974d8e8a4e998d241bb12f', 'request_id': 'req-adaf33e3-1c9e-470a-badb-61c7995d9cec'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.225226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] The token cache is being invalidate because project d3f965d3d5974d8e8a4e998d241bb12f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:35.226044 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.227185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.229121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 971/1929] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:34 2022] DELETE /identity/v3/projects/d3f965d3d5974d8e8a4e998d241bb12f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:35.233058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:35.234587 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:35.249007 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e4f05-e792-476b-aceb-a1baf61ef98c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:35.320158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:35.320158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:35.328084 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3cd08-d074-4209-a219-f4396b301cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:35.329048 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3cd08-d074-4209-a219-f4396b301cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:35.329396 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3cd08-d074-4209-a219-f4396b301cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:35.332347 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3cd08-d074-4209-a219-f4396b301cff admin admin] PATH_INFO: `/v3/users/36206dcab9a04205a62f9a9a0a8ae444` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:35.333932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3cd08-d074-4209-a219-f4396b301cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=36206dcab9a04205a62f9a9a0a8ae444)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:35.337043 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3cd08-d074-4209-a219-f4396b301cff admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:35.341788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaf33e3-1c9e-470a-badb-61c7995d9cec admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:35.356581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc99a98-e267-4050-bc0f-15dc6ed0f305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:35.357242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc99a98-e267-4050-bc0f-15dc6ed0f305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:35.357518 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc99a98-e267-4050-bc0f-15dc6ed0f305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:35.358134 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc99a98-e267-4050-bc0f-15dc6ed0f305 admin admin] PATH_INFO: `/v3/projects/8b7907bc40c44973b8073a37eb039489` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:35.363322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc99a98-e267-4050-bc0f-15dc6ed0f305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b7907bc40c44973b8073a37eb039489)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:35.366957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc99a98-e267-4050-bc0f-15dc6ed0f305 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:35.574521 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2da3cd08-d074-4209-a219-f4396b301cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36206dcab9a04205a62f9a9a0a8ae444'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.575597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2da3cd08-d074-4209-a219-f4396b301cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36206dcab9a04205a62f9a9a0a8ae444'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.582138 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-bfc99a98-e267-4050-bc0f-15dc6ed0f305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b7907bc40c44973b8073a37eb039489', 'request_id': 'req-bfc99a98-e267-4050-bc0f-15dc6ed0f305'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.588636 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-bfc99a98-e267-4050-bc0f-15dc6ed0f305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b7907bc40c44973b8073a37eb039489', 'request_id': 'req-bfc99a98-e267-4050-bc0f-15dc6ed0f305'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.595775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-bfc99a98-e267-4050-bc0f-15dc6ed0f305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b7907bc40c44973b8073a37eb039489', 'request_id': 'req-bfc99a98-e267-4050-bc0f-15dc6ed0f305'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.682989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2da3cd08-d074-4209-a219-f4396b301cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36206dcab9a04205a62f9a9a0a8ae444'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.706693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2da3cd08-d074-4209-a219-f4396b301cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36206dcab9a04205a62f9a9a0a8ae444'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.712029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2da3cd08-d074-4209-a219-f4396b301cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36206dcab9a04205a62f9a9a0a8ae444'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.739074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 959/1930] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:35 2022] DELETE /identity/v3/users/36206dcab9a04205a62f9a9a0a8ae444 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:35.768248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-bfc99a98-e267-4050-bc0f-15dc6ed0f305 admin admin] The token cache is being invalidate because project 8b7907bc40c44973b8073a37eb039489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:35.768557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-bfc99a98-e267-4050-bc0f-15dc6ed0f305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.774467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-bfc99a98-e267-4050-bc0f-15dc6ed0f305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:35.774467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 972/1931] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:35 2022] DELETE /identity/v3/projects/8b7907bc40c44973b8073a37eb039489 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:35.777188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e693b-3812-4ca8-8eb6-d042049eae8a None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:35.777478 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e693b-3812-4ca8-8eb6-d042049eae8a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:35.777886 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e693b-3812-4ca8-8eb6-d042049eae8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:35.802509 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-fd2e693b-3812-4ca8-8eb6-d042049eae8a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:35.804191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-fd2e693b-3812-4ca8-8eb6-d042049eae8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:35.900817 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 960/1932] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:35.908721 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd24cc1-67bb-47dc-adf0-c2d306cd5118 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:35.909205 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd24cc1-67bb-47dc-adf0-c2d306cd5118 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:35.909585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd24cc1-67bb-47dc-adf0-c2d306cd5118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:35.935118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-cbd24cc1-67bb-47dc-adf0-c2d306cd5118 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:35.936699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-cbd24cc1-67bb-47dc-adf0-c2d306cd5118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:35.952521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 973/1933] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:35.957459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e693b-3812-4ca8-8eb6-d042049eae8a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:35.968198 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e693b-3812-4ca8-8eb6-d042049eae8a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:35.974869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973ee55-88e2-4e52-91cf-28f0307f25ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:35.975508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973ee55-88e2-4e52-91cf-28f0307f25ae admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:35.975787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973ee55-88e2-4e52-91cf-28f0307f25ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:35.976058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973ee55-88e2-4e52-91cf-28f0307f25ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:35.976713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973ee55-88e2-4e52-91cf-28f0307f25ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:43:35.977015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973ee55-88e2-4e52-91cf-28f0307f25ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:35.979937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4973ee55-88e2-4e52-91cf-28f0307f25ae admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:35.990206 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:35.997175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 961/1934] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:43:36.001290 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd24cc1-67bb-47dc-adf0-c2d306cd5118 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:36.012061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd24cc1-67bb-47dc-adf0-c2d306cd5118 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:36.017090 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09232f16-d8fb-4a5f-9f73-9c96780342b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:36.017765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09232f16-d8fb-4a5f-9f73-9c96780342b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:36.021003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09232f16-d8fb-4a5f-9f73-9c96780342b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:36.021003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09232f16-d8fb-4a5f-9f73-9c96780342b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:36.021003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09232f16-d8fb-4a5f-9f73-9c96780342b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:36.025839 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09232f16-d8fb-4a5f-9f73-9c96780342b1 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:36.146901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 974/1935] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:36 2022] POST /identity/v3/projects => generated 409 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:36.147864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973ee55-88e2-4e52-91cf-28f0307f25ae admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:36.161912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4973ee55-88e2-4e52-91cf-28f0307f25ae admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:36.228982 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d7736-3e61-4f31-99e4-bbe1b0ded783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:36.229621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d7736-3e61-4f31-99e4-bbe1b0ded783 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:36.229945 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d7736-3e61-4f31-99e4-bbe1b0ded783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:36.230239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d7736-3e61-4f31-99e4-bbe1b0ded783 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:36.231298 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d7736-3e61-4f31-99e4-bbe1b0ded783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:36.233985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d7736-3e61-4f31-99e4-bbe1b0ded783 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:36.345927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 962/1936] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:36 2022] POST /identity/v3/users => generated 305 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:43:36.350538 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09232f16-d8fb-4a5f-9f73-9c96780342b1 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:36.364268 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09232f16-d8fb-4a5f-9f73-9c96780342b1 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:36.369252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55433e2a-b45c-4643-baf3-0630f24e53f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:36.374694 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55433e2a-b45c-4643-baf3-0630f24e53f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:36.374694 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55433e2a-b45c-4643-baf3-0630f24e53f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:36.374694 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55433e2a-b45c-4643-baf3-0630f24e53f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:36.374694 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55433e2a-b45c-4643-baf3-0630f24e53f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:36.374694 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55433e2a-b45c-4643-baf3-0630f24e53f9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:36.385404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 975/1937] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:36.394357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d7736-3e61-4f31-99e4-bbe1b0ded783 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:36.408314 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d7736-3e61-4f31-99e4-bbe1b0ded783 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:36.413957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1cd47-a676-43dd-b560-b6b3576c3b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:36.414782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1cd47-a676-43dd-b560-b6b3576c3b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:36.415131 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1cd47-a676-43dd-b560-b6b3576c3b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:36.416017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1cd47-a676-43dd-b560-b6b3576c3b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:36.416250 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1cd47-a676-43dd-b560-b6b3576c3b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:36.419016 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1cd47-a676-43dd-b560-b6b3576c3b05 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:36.434096 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 963/1938] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:43:36.438483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55433e2a-b45c-4643-baf3-0630f24e53f9 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:36.449604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55433e2a-b45c-4643-baf3-0630f24e53f9 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:36.456008 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826cf6d-7f49-4331-b413-0980c5c13ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:36.456681 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826cf6d-7f49-4331-b413-0980c5c13ada admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:36.456950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826cf6d-7f49-4331-b413-0980c5c13ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:36.457222 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826cf6d-7f49-4331-b413-0980c5c13ada admin admin] PATH_INFO: `/v3/projects/bf8299edcc474c43b7613d1d2c915c4a/users/6beb52c7292d491cbb60f5474beb7f95/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:36.469849 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:36.474925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826cf6d-7f49-4331-b413-0980c5c13ada admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf8299edcc474c43b7613d1d2c915c4a, user_id=6beb52c7292d491cbb60f5474beb7f95, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:36.477802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7826cf6d-7f49-4331-b413-0980c5c13ada admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:36.498323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1cd47-a676-43dd-b560-b6b3576c3b05 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:36.500161 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8eb1cd47-a676-43dd-b560-b6b3576c3b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:36.522589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1cd47-a676-43dd-b560-b6b3576c3b05 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:36.530496 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2bc9c-68f4-417c-b94d-1fa0cc041954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:36.531656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2bc9c-68f4-417c-b94d-1fa0cc041954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:36.532268 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2bc9c-68f4-417c-b94d-1fa0cc041954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:36.532786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2bc9c-68f4-417c-b94d-1fa0cc041954 admin admin] PATH_INFO: `/v3/projects/c4e7765628f842b194c26177dc40d32a` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:36.545688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2bc9c-68f4-417c-b94d-1fa0cc041954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e7765628f842b194c26177dc40d32a)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:36.549105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2bc9c-68f4-417c-b94d-1fa0cc041954 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:36.595775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 976/1939] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:36 2022] PUT /identity/v3/projects/bf8299edcc474c43b7613d1d2c915c4a/users/6beb52c7292d491cbb60f5474beb7f95/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:36.604366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182d0a3-d692-45f9-beb7-36e181df4aea None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:36.604366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182d0a3-d692-45f9-beb7-36e181df4aea None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:36.604366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182d0a3-d692-45f9-beb7-36e181df4aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:36.642172 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-7182d0a3-d692-45f9-beb7-36e181df4aea None None] MFA Rules not processed for user `6beb52c7292d491cbb60f5474beb7f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:36.642172 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7182d0a3-d692-45f9-beb7-36e181df4aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:36.741648 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 977/1940] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:36 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:36.745612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182d0a3-d692-45f9-beb7-36e181df4aea None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:36.746525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-81f2bc9c-68f4-417c-b94d-1fa0cc041954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e7765628f842b194c26177dc40d32a', 'request_id': 'req-81f2bc9c-68f4-417c-b94d-1fa0cc041954'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:36.773850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-81f2bc9c-68f4-417c-b94d-1fa0cc041954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e7765628f842b194c26177dc40d32a', 'request_id': 'req-81f2bc9c-68f4-417c-b94d-1fa0cc041954'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:36.782449 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-81f2bc9c-68f4-417c-b94d-1fa0cc041954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e7765628f842b194c26177dc40d32a', 'request_id': 'req-81f2bc9c-68f4-417c-b94d-1fa0cc041954'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:36.867296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-81f2bc9c-68f4-417c-b94d-1fa0cc041954 admin admin] The token cache is being invalidate because project c4e7765628f842b194c26177dc40d32a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:36.867688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-81f2bc9c-68f4-417c-b94d-1fa0cc041954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:36.869470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-81f2bc9c-68f4-417c-b94d-1fa0cc041954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:36.872203 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 964/1941] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:36 2022] DELETE /identity/v3/projects/c4e7765628f842b194c26177dc40d32a => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:36.877212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07f0a3-ab9f-46c6-85a4-d34b7dd47897 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:36.877702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07f0a3-ab9f-46c6-85a4-d34b7dd47897 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:36.878208 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07f0a3-ab9f-46c6-85a4-d34b7dd47897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:36.900417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182d0a3-d692-45f9-beb7-36e181df4aea None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:36.919592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-1b07f0a3-ab9f-46c6-85a4-d34b7dd47897 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:36.921973 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1b07f0a3-ab9f-46c6-85a4-d34b7dd47897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:37.025274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:37.056480 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 965/1942] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:37.059923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278b7fd-2676-4dbc-b360-ea334063fdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:37.060731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278b7fd-2676-4dbc-b360-ea334063fdd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:37.061058 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278b7fd-2676-4dbc-b360-ea334063fdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:37.061373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278b7fd-2676-4dbc-b360-ea334063fdd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:37.062212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278b7fd-2676-4dbc-b360-ea334063fdd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:37.064168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d257b47-9c3b-4e5a-9f72-86750bb2bcd1 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:37.065535 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d257b47-9c3b-4e5a-9f72-86750bb2bcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:37.066113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d257b47-9c3b-4e5a-9f72-86750bb2bcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:37.071951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278b7fd-2676-4dbc-b360-ea334063fdd0 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:37.106905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-7d257b47-9c3b-4e5a-9f72-86750bb2bcd1 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:37.108177 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7d257b47-9c3b-4e5a-9f72-86750bb2bcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:37.126522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 966/1943] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:37.131760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d257b47-9c3b-4e5a-9f72-86750bb2bcd1 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:37.142456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d257b47-9c3b-4e5a-9f72-86750bb2bcd1 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:37.148580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019e411-b993-4155-a938-522d9dfad876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:37.149004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019e411-b993-4155-a938-522d9dfad876 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:37.149348 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019e411-b993-4155-a938-522d9dfad876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:37.149686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019e411-b993-4155-a938-522d9dfad876 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:37.150514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019e411-b993-4155-a938-522d9dfad876 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:43:37.150824 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019e411-b993-4155-a938-522d9dfad876 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:37.152610 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019e411-b993-4155-a938-522d9dfad876 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:37.169757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 967/1944] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:43:37.179608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019e411-b993-4155-a938-522d9dfad876 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:37.198052 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 978/1945] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:36 2022] POST /identity/v3/projects => generated 405 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:43:37.201677 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019e411-b993-4155-a938-522d9dfad876 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:37.203098 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278b7fd-2676-4dbc-b360-ea334063fdd0 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:37.208918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7278b7fd-2676-4dbc-b360-ea334063fdd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:37.247034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:37.367418 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278b7fd-2676-4dbc-b360-ea334063fdd0 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:37.368388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a575b-00f4-44f1-8e1b-ea2f50be3892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:37.369195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a575b-00f4-44f1-8e1b-ea2f50be3892 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:37.369532 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a575b-00f4-44f1-8e1b-ea2f50be3892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:37.369816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a575b-00f4-44f1-8e1b-ea2f50be3892 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:37.370471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a575b-00f4-44f1-8e1b-ea2f50be3892 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:37.372403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a575b-00f4-44f1-8e1b-ea2f50be3892 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:37.382588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af341a-3a4b-48d6-a502-900a5e307f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:37.384732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af341a-3a4b-48d6-a502-900a5e307f1f admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:37.385053 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af341a-3a4b-48d6-a502-900a5e307f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:37.385325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af341a-3a4b-48d6-a502-900a5e307f1f admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:37.386365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af341a-3a4b-48d6-a502-900a5e307f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:37.388313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af341a-3a4b-48d6-a502-900a5e307f1f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:37.507041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 968/1946] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:37 2022] POST /identity/v3/projects => generated 365 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:37.508505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a575b-00f4-44f1-8e1b-ea2f50be3892 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:37.591355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 979/1947] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:37 2022] POST /identity/v3/users => generated 309 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:43:37.594925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af341a-3a4b-48d6-a502-900a5e307f1f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:37.610295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af341a-3a4b-48d6-a502-900a5e307f1f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:37.698968 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a575b-00f4-44f1-8e1b-ea2f50be3892 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:37.730001 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ea905-b1d2-4d60-ba2b-5660be9f0a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:37.730706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ea905-b1d2-4d60-ba2b-5660be9f0a84 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:37.730982 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ea905-b1d2-4d60-ba2b-5660be9f0a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:37.731333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ea905-b1d2-4d60-ba2b-5660be9f0a84 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:37.732053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ea905-b1d2-4d60-ba2b-5660be9f0a84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:37.735342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ea905-b1d2-4d60-ba2b-5660be9f0a84 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:37.821729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73046779-8f15-4b56-8232-631c56e1e855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:37.822396 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73046779-8f15-4b56-8232-631c56e1e855 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:37.822675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73046779-8f15-4b56-8232-631c56e1e855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:37.822975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73046779-8f15-4b56-8232-631c56e1e855 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:37.826039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73046779-8f15-4b56-8232-631c56e1e855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:37.835411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73046779-8f15-4b56-8232-631c56e1e855 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:37.865586 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 969/1948] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:37 2022] POST /identity/v3/users => generated 283 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:37.869285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ea905-b1d2-4d60-ba2b-5660be9f0a84 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:37.874069 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 980/1949] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:37.879625 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73046779-8f15-4b56-8232-631c56e1e855 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:37.892100 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ea905-b1d2-4d60-ba2b-5660be9f0a84 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:37.895671 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73046779-8f15-4b56-8232-631c56e1e855 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:37.901967 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55828c97-d973-4528-ad13-9bdd92383089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:37.902937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55828c97-d973-4528-ad13-9bdd92383089 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:37.903480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55828c97-d973-4528-ad13-9bdd92383089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:37.903857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55828c97-d973-4528-ad13-9bdd92383089 admin admin] PATH_INFO: `/v3/projects/8a61676df1fe41cab96bbe3fce56af35/users/132824f9284b471cac6cb850815529c1/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:37.905544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c1183-e097-4744-b1c8-1f42b8b8d84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:37.906360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c1183-e097-4744-b1c8-1f42b8b8d84e admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:37.906783 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c1183-e097-4744-b1c8-1f42b8b8d84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:37.907219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c1183-e097-4744-b1c8-1f42b8b8d84e admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:37.908015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c1183-e097-4744-b1c8-1f42b8b8d84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:37.926239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c1183-e097-4744-b1c8-1f42b8b8d84e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:37.937020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55828c97-d973-4528-ad13-9bdd92383089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a61676df1fe41cab96bbe3fce56af35, user_id=132824f9284b471cac6cb850815529c1, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:37.948414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55828c97-d973-4528-ad13-9bdd92383089 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:37.963714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 970/1950] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:37.973964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c1183-e097-4744-b1c8-1f42b8b8d84e admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:37.990994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c1183-e097-4744-b1c8-1f42b8b8d84e admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:37.999431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178cc9c-859d-4205-b85a-c68c55a534de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:38.000621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178cc9c-859d-4205-b85a-c68c55a534de admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:38.001250 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178cc9c-859d-4205-b85a-c68c55a534de admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:38.001878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178cc9c-859d-4205-b85a-c68c55a534de admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:38.008385 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178cc9c-859d-4205-b85a-c68c55a534de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:38.008385 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178cc9c-859d-4205-b85a-c68c55a534de admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:38.027377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 971/1951] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:38.034095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178cc9c-859d-4205-b85a-c68c55a534de admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:38.034947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 981/1952] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:37 2022] PUT /identity/v3/projects/8a61676df1fe41cab96bbe3fce56af35/users/132824f9284b471cac6cb850815529c1/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:38.039518 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55828c97-d973-4528-ad13-9bdd92383089 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:38.050826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178cc9c-859d-4205-b85a-c68c55a534de admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:38.099507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:38.148245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a15f1-bd8b-4c45-ac71-ac8e482b1b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:38.148949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a15f1-bd8b-4c45-ac71-ac8e482b1b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:38.149616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a15f1-bd8b-4c45-ac71-ac8e482b1b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:38.149616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a15f1-bd8b-4c45-ac71-ac8e482b1b33 admin admin] PATH_INFO: `/v3/projects/035a55a6a0684843b995b2bb854f5f99/users/662fa45e9ffd48eda58470ceeac3866d/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:38.163728 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55828c97-d973-4528-ad13-9bdd92383089 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:38.174788 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a15f1-bd8b-4c45-ac71-ac8e482b1b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=035a55a6a0684843b995b2bb854f5f99, user_id=662fa45e9ffd48eda58470ceeac3866d, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:38.177873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a15f1-bd8b-4c45-ac71-ac8e482b1b33 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:38.178122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2e472-0dc3-42c2-85cb-5a6b7e41ccc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:38.178869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2e472-0dc3-42c2-85cb-5a6b7e41ccc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:38.179263 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2e472-0dc3-42c2-85cb-5a6b7e41ccc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:38.179637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2e472-0dc3-42c2-85cb-5a6b7e41ccc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:38.180466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2e472-0dc3-42c2-85cb-5a6b7e41ccc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:38.184626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2e472-0dc3-42c2-85cb-5a6b7e41ccc3 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:38.219785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 982/1953] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:38.224740 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2e472-0dc3-42c2-85cb-5a6b7e41ccc3 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:38.269049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 972/1954] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:38 2022] PUT /identity/v3/projects/035a55a6a0684843b995b2bb854f5f99/users/662fa45e9ffd48eda58470ceeac3866d/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:38.272353 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2e472-0dc3-42c2-85cb-5a6b7e41ccc3 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:38.283173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9984b60-6681-4fa3-bef1-651a3f45201e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:38.283513 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9984b60-6681-4fa3-bef1-651a3f45201e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:38.284335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9984b60-6681-4fa3-bef1-651a3f45201e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:38.288577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:38.343076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c9984b60-6681-4fa3-bef1-651a3f45201e None None] MFA Rules not processed for user `662fa45e9ffd48eda58470ceeac3866d`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:38.344455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c9984b60-6681-4fa3-bef1-651a3f45201e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:38.395195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82913f-ca4e-40d8-8660-06f822eaa9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:38.396265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82913f-ca4e-40d8-8660-06f822eaa9f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:38.396765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82913f-ca4e-40d8-8660-06f822eaa9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:38.397141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82913f-ca4e-40d8-8660-06f822eaa9f8 admin admin] PATH_INFO: `/v3/projects/8a61676df1fe41cab96bbe3fce56af35/users/132824f9284b471cac6cb850815529c1/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:38.399258 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82913f-ca4e-40d8-8660-06f822eaa9f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a61676df1fe41cab96bbe3fce56af35, user_id=132824f9284b471cac6cb850815529c1, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:38.402231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82913f-ca4e-40d8-8660-06f822eaa9f8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:38.423231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 983/1955] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:38 2022] PUT /identity/v3/projects/8a61676df1fe41cab96bbe3fce56af35/users/132824f9284b471cac6cb850815529c1/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:38.434539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba026fb-db22-4045-9acc-374cf7873085 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:38.434539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba026fb-db22-4045-9acc-374cf7873085 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:38.439720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba026fb-db22-4045-9acc-374cf7873085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:38.481588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 973/1956] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:38.487739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9984b60-6681-4fa3-bef1-651a3f45201e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:38.497571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-aba026fb-db22-4045-9acc-374cf7873085 None None] MFA Rules not processed for user `132824f9284b471cac6cb850815529c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:38.498843 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-aba026fb-db22-4045-9acc-374cf7873085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:38.505411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9984b60-6681-4fa3-bef1-651a3f45201e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:38.630716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8355acb-dc8b-4006-8250-fa552fac2fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:38.631367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8355acb-dc8b-4006-8250-fa552fac2fee admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:38.631755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8355acb-dc8b-4006-8250-fa552fac2fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:38.632028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8355acb-dc8b-4006-8250-fa552fac2fee admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:38.632675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8355acb-dc8b-4006-8250-fa552fac2fee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:38.636974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8355acb-dc8b-4006-8250-fa552fac2fee admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:38.686087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 984/1957] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:38 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:38.735297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba026fb-db22-4045-9acc-374cf7873085 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:38.736216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-aba026fb-db22-4045-9acc-374cf7873085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:38.802027 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 974/1958] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:38 2022] POST /identity/v3/projects => generated 363 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:38.806751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8355acb-dc8b-4006-8250-fa552fac2fee admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:38.839684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba026fb-db22-4045-9acc-374cf7873085 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:38.870315 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:38.907489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:38.915608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8355acb-dc8b-4006-8250-fa552fac2fee admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:38.923516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5872f0c-7652-467d-9614-542bf93697b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:38.924976 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5872f0c-7652-467d-9614-542bf93697b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:38.925356 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5872f0c-7652-467d-9614-542bf93697b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:38.925910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5872f0c-7652-467d-9614-542bf93697b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:38.927086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5872f0c-7652-467d-9614-542bf93697b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:38.933605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5872f0c-7652-467d-9614-542bf93697b2 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:38.935606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8dc67-6cbf-450f-8ae1-176688f97cb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:38.936579 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8dc67-6cbf-450f-8ae1-176688f97cb0 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:38.937085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8dc67-6cbf-450f-8ae1-176688f97cb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:38.937538 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8dc67-6cbf-450f-8ae1-176688f97cb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:39.031449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8dc67-6cbf-450f-8ae1-176688f97cb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:39.032749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8dc67-6cbf-450f-8ae1-176688f97cb0 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:39.050102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 985/1959] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:38 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:43:39.065919 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 975/1960] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:38 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:43:39.068441 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8dc67-6cbf-450f-8ae1-176688f97cb0 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:39.084111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8dc67-6cbf-450f-8ae1-176688f97cb0 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:39.090176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13322f69-77ca-4c36-9183-fbf89b83e086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:39.090784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13322f69-77ca-4c36-9183-fbf89b83e086 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:39.091065 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13322f69-77ca-4c36-9183-fbf89b83e086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:39.091333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13322f69-77ca-4c36-9183-fbf89b83e086 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:39.092616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13322f69-77ca-4c36-9183-fbf89b83e086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:39.094300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13322f69-77ca-4c36-9183-fbf89b83e086 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:39.125523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 986/1961] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:39.133607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5872f0c-7652-467d-9614-542bf93697b2 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:39.144935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5872f0c-7652-467d-9614-542bf93697b2 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:39.150310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04770b3f-19af-446a-b558-58c480dbda99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:39.151184 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04770b3f-19af-446a-b558-58c480dbda99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:39.151461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04770b3f-19af-446a-b558-58c480dbda99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:39.151783 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04770b3f-19af-446a-b558-58c480dbda99 admin admin] PATH_INFO: `/v3/projects/02b0007126c044ed9663c8d19cca92a7/users/b07f4bcd99284e4babc1d80d27c04d99/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:39.168461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04770b3f-19af-446a-b558-58c480dbda99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b0007126c044ed9663c8d19cca92a7, user_id=b07f4bcd99284e4babc1d80d27c04d99, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:39.171167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04770b3f-19af-446a-b558-58c480dbda99 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:39.220315 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 976/1962] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:39 2022] PUT /identity/v3/projects/02b0007126c044ed9663c8d19cca92a7/users/b07f4bcd99284e4babc1d80d27c04d99/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:39.225901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13322f69-77ca-4c36-9183-fbf89b83e086 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:39.254920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13322f69-77ca-4c36-9183-fbf89b83e086 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:39.351159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d054d04-cfa1-41df-b85b-abf2ff8a4f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:39.351783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d054d04-cfa1-41df-b85b-abf2ff8a4f02 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:39.352063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d054d04-cfa1-41df-b85b-abf2ff8a4f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:39.352339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d054d04-cfa1-41df-b85b-abf2ff8a4f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:39.353047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d054d04-cfa1-41df-b85b-abf2ff8a4f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:39.355128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d054d04-cfa1-41df-b85b-abf2ff8a4f02 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:39.371736 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 987/1963] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:39.375893 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04770b3f-19af-446a-b558-58c480dbda99 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:39.388007 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04770b3f-19af-446a-b558-58c480dbda99 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:39.400292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75e351-e959-4ffe-9bcc-5f69611715e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:39.402282 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75e351-e959-4ffe-9bcc-5f69611715e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:39.403523 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75e351-e959-4ffe-9bcc-5f69611715e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:39.403984 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75e351-e959-4ffe-9bcc-5f69611715e6 admin admin] PATH_INFO: `/v3/projects/02b0007126c044ed9663c8d19cca92a7/users/b07f4bcd99284e4babc1d80d27c04d99/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:39.406912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75e351-e959-4ffe-9bcc-5f69611715e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b0007126c044ed9663c8d19cca92a7, user_id=b07f4bcd99284e4babc1d80d27c04d99, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:39.411567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75e351-e959-4ffe-9bcc-5f69611715e6 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:39.434811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 977/1964] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:39 2022] PUT /identity/v3/projects/02b0007126c044ed9663c8d19cca92a7/users/b07f4bcd99284e4babc1d80d27c04d99/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:39.437611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee5b69-680f-4c17-868e-19fb726c8a2f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:39.438149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee5b69-680f-4c17-868e-19fb726c8a2f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:39.438693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee5b69-680f-4c17-868e-19fb726c8a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:39.472466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a2ee5b69-680f-4c17-868e-19fb726c8a2f None None] MFA Rules not processed for user `b07f4bcd99284e4babc1d80d27c04d99`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:39.474051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a2ee5b69-680f-4c17-868e-19fb726c8a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:39.491789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75e351-e959-4ffe-9bcc-5f69611715e6 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:39.570215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 988/1965] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:39.583065 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee5b69-680f-4c17-868e-19fb726c8a2f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:39.585903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a2ee5b69-680f-4c17-868e-19fb726c8a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:39.595905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:39.627123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75e351-e959-4ffe-9bcc-5f69611715e6 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:39.632866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdca91d-f8b7-4b17-bc31-e3c6728f9b96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:39.635381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdca91d-f8b7-4b17-bc31-e3c6728f9b96 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:39.635381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdca91d-f8b7-4b17-bc31-e3c6728f9b96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:39.635381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdca91d-f8b7-4b17-bc31-e3c6728f9b96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:39.636755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5fdca91d-f8b7-4b17-bc31-e3c6728f9b96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:39.664924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee5b69-680f-4c17-868e-19fb726c8a2f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:39.669372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51055c49-8ab2-4563-a690-7be2b491cf93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:39.670028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51055c49-8ab2-4563-a690-7be2b491cf93 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:39.670310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51055c49-8ab2-4563-a690-7be2b491cf93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:39.672793 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51055c49-8ab2-4563-a690-7be2b491cf93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:39.687349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51055c49-8ab2-4563-a690-7be2b491cf93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:39.688924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51055c49-8ab2-4563-a690-7be2b491cf93 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:39.702128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 989/1966] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:43:39.732592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdca91d-f8b7-4b17-bc31-e3c6728f9b96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:39.732592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdca91d-f8b7-4b17-bc31-e3c6728f9b96 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:39.738624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 978/1967] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:39 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:43:39.997861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51055c49-8ab2-4563-a690-7be2b491cf93 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:40.008576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51055c49-8ab2-4563-a690-7be2b491cf93 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:40.012426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506798b-1e89-4049-9963-2828c8cdc39e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:40.013034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506798b-1e89-4049-9963-2828c8cdc39e service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:40.013310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506798b-1e89-4049-9963-2828c8cdc39e service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:40.013674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506798b-1e89-4049-9963-2828c8cdc39e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:40.078854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506798b-1e89-4049-9963-2828c8cdc39e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:40.080600 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c506798b-1e89-4049-9963-2828c8cdc39e service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:40.091298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 990/1968] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:41.267417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dde0d5-b05f-4fd8-93db-05dfa45ca774 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:41.267943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dde0d5-b05f-4fd8-93db-05dfa45ca774 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:41.268116 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dde0d5-b05f-4fd8-93db-05dfa45ca774 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:41.269548 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 979/1969] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:43:41.297547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a7b43-a54d-4365-8801-ac35cc378284 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:41.298119 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a7b43-a54d-4365-8801-ac35cc378284 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:41.299948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a7b43-a54d-4365-8801-ac35cc378284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:41.321630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-236a7b43-a54d-4365-8801-ac35cc378284 None None] MFA Rules not processed for user `9975c0dab76f4ccbb4d2107e8e780ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:41.322903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-236a7b43-a54d-4365-8801-ac35cc378284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:41.396014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 991/1970] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:43:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:41.439350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51e7d5-a3b6-40af-84b2-a48a68b72e12 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:41.439628 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51e7d5-a3b6-40af-84b2-a48a68b72e12 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:41.439926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51e7d5-a3b6-40af-84b2-a48a68b72e12 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:41.441753 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 980/1971] 10.222.0.49 () {54 vars in 961 bytes} [Fri Sep 16 05:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:43:41.447250 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500b99e-a779-49eb-bf7f-b94cf6902e44 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:41.447535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500b99e-a779-49eb-bf7f-b94cf6902e44 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:41.447803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500b99e-a779-49eb-bf7f-b94cf6902e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:41.474760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-3500b99e-a779-49eb-bf7f-b94cf6902e44 None None] MFA Rules not processed for user `9975c0dab76f4ccbb4d2107e8e780ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:41.476150 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3500b99e-a779-49eb-bf7f-b94cf6902e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:41.485887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 992/1972] 10.222.0.49 () {58 vars in 1058 bytes} [Fri Sep 16 05:43:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:41.493366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51e7d5-a3b6-40af-84b2-a48a68b72e12 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:41.507893 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51e7d5-a3b6-40af-84b2-a48a68b72e12 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:41.512997 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e1298-de02-4f42-9bdf-7b5cdeff29cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9975c0dab76f4ccbb4d2107e8e780ae5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:41.513548 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e1298-de02-4f42-9bdf-7b5cdeff29cf service placement] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:41.513835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e1298-de02-4f42-9bdf-7b5cdeff29cf service placement] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:41.514111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e1298-de02-4f42-9bdf-7b5cdeff29cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:41.522706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e1298-de02-4f42-9bdf-7b5cdeff29cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:41.524408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e1298-de02-4f42-9bdf-7b5cdeff29cf service placement] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:41.535068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 981/1973] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:42.117550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777bbfd-d1bd-43ca-9c64-c96eaf2c447f None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:42.117846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777bbfd-d1bd-43ca-9c64-c96eaf2c447f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:42.118150 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777bbfd-d1bd-43ca-9c64-c96eaf2c447f None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:42.119552 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 993/1974] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:43:42.158884 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fb663-f72b-44a4-b118-f738cde60876 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:42.159170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fb663-f72b-44a4-b118-f738cde60876 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:42.159448 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fb663-f72b-44a4-b118-f738cde60876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:42.183703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-b31fb663-f72b-44a4-b118-f738cde60876 None None] MFA Rules not processed for user `9975c0dab76f4ccbb4d2107e8e780ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:42.184979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b31fb663-f72b-44a4-b118-f738cde60876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:42.194569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 982/1975] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:43:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:42.210133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777bbfd-d1bd-43ca-9c64-c96eaf2c447f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:42.224079 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777bbfd-d1bd-43ca-9c64-c96eaf2c447f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:42.229076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aadd5c-7dad-4ad2-98cd-af276400c295 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9975c0dab76f4ccbb4d2107e8e780ae5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:42.229668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aadd5c-7dad-4ad2-98cd-af276400c295 service placement] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:42.229973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aadd5c-7dad-4ad2-98cd-af276400c295 service placement] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:42.230240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aadd5c-7dad-4ad2-98cd-af276400c295 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:42.240307 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aadd5c-7dad-4ad2-98cd-af276400c295 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:42.242040 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aadd5c-7dad-4ad2-98cd-af276400c295 service placement] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:42.255699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 994/1976] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:42.264994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fb663-f72b-44a4-b118-f738cde60876 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:42.275406 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b31fb663-f72b-44a4-b118-f738cde60876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:42.296828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:42.345408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fb663-f72b-44a4-b118-f738cde60876 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:42.350870 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:42.351495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:42.351766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:42.352027 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] PATH_INFO: `/v3/users/b29bb91e157048af8fea6eead061bd1d` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:42.353136 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b29bb91e157048af8fea6eead061bd1d)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:42.355059 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:42.547212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b29bb91e157048af8fea6eead061bd1d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:42.547853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b29bb91e157048af8fea6eead061bd1d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:42.630361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b29bb91e157048af8fea6eead061bd1d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:42.637946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b29bb91e157048af8fea6eead061bd1d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:42.638619 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b29bb91e157048af8fea6eead061bd1d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:42.656744 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 983/1977] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:42 2022] DELETE /identity/v3/users/b29bb91e157048af8fea6eead061bd1d => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:42.918105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aadd5c-7dad-4ad2-98cd-af276400c295 service placement] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:42.929843 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aadd5c-7dad-4ad2-98cd-af276400c295 service placement] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:42.949687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:42.990200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:42.990827 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:42.991104 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:42.991374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] PATH_INFO: `/v3/projects/9c4cc7113aa341a5bc8e40d03d2026e6` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:42.992477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4cc7113aa341a5bc8e40d03d2026e6)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:42.994402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:43.094597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4cc7113aa341a5bc8e40d03d2026e6', 'request_id': 'req-c9a5e861-3612-4687-bb49-556ed4ca25b8'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:43.100398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4cc7113aa341a5bc8e40d03d2026e6', 'request_id': 'req-c9a5e861-3612-4687-bb49-556ed4ca25b8'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:43.106560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4cc7113aa341a5bc8e40d03d2026e6', 'request_id': 'req-c9a5e861-3612-4687-bb49-556ed4ca25b8'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:43.179793 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] The token cache is being invalidate because project 9c4cc7113aa341a5bc8e40d03d2026e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:43.180080 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:43.180827 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:43.182506 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 995/1978] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:42 2022] DELETE /identity/v3/projects/9c4cc7113aa341a5bc8e40d03d2026e6 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:43.187616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:43.188558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:43.202686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59857f-cf0e-42d5-ab50-d47511113a83 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:43.271164 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d31e92-6293-4450-9a4c-1a4f0c9f1ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:43.271825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d31e92-6293-4450-9a4c-1a4f0c9f1ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:43.272100 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d31e92-6293-4450-9a4c-1a4f0c9f1ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:43.272367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d31e92-6293-4450-9a4c-1a4f0c9f1ea4 admin admin] PATH_INFO: `/v3/users/987b20939db94554b6226728a38f8e61` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:43.273477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d31e92-6293-4450-9a4c-1a4f0c9f1ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=987b20939db94554b6226728a38f8e61)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:43.275470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d31e92-6293-4450-9a4c-1a4f0c9f1ea4 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:43.501259 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:43.508607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-f2d31e92-6293-4450-9a4c-1a4f0c9f1ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '987b20939db94554b6226728a38f8e61'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:43.508687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-f2d31e92-6293-4450-9a4c-1a4f0c9f1ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '987b20939db94554b6226728a38f8e61'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:43.585344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-f2d31e92-6293-4450-9a4c-1a4f0c9f1ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '987b20939db94554b6226728a38f8e61'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:43.592785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-f2d31e92-6293-4450-9a4c-1a4f0c9f1ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '987b20939db94554b6226728a38f8e61'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:43.593400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-f2d31e92-6293-4450-9a4c-1a4f0c9f1ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '987b20939db94554b6226728a38f8e61'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:43.611338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 984/1979] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:43 2022] DELETE /identity/v3/users/987b20939db94554b6226728a38f8e61 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:43.849233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:43.850741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:43.864261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5e861-3612-4687-bb49-556ed4ca25b8 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:43.921004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be880ac5-a82a-4297-b20f-7d6118435c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:43.921639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be880ac5-a82a-4297-b20f-7d6118435c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:43.921923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be880ac5-a82a-4297-b20f-7d6118435c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:43.922191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be880ac5-a82a-4297-b20f-7d6118435c21 admin admin] PATH_INFO: `/v3/projects/9d2e4b0153ff4563b91a906926526018` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:43.923305 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be880ac5-a82a-4297-b20f-7d6118435c21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d2e4b0153ff4563b91a906926526018)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:43.925406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be880ac5-a82a-4297-b20f-7d6118435c21 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:44.031020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-be880ac5-a82a-4297-b20f-7d6118435c21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d2e4b0153ff4563b91a906926526018', 'request_id': 'req-be880ac5-a82a-4297-b20f-7d6118435c21'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:44.035616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:44.037697 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-be880ac5-a82a-4297-b20f-7d6118435c21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d2e4b0153ff4563b91a906926526018', 'request_id': 'req-be880ac5-a82a-4297-b20f-7d6118435c21'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:44.043869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-be880ac5-a82a-4297-b20f-7d6118435c21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d2e4b0153ff4563b91a906926526018', 'request_id': 'req-be880ac5-a82a-4297-b20f-7d6118435c21'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:44.138791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-be880ac5-a82a-4297-b20f-7d6118435c21 admin admin] The token cache is being invalidate because project 9d2e4b0153ff4563b91a906926526018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:44.139095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-be880ac5-a82a-4297-b20f-7d6118435c21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:44.139703 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-be880ac5-a82a-4297-b20f-7d6118435c21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:44.141155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 996/1980] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:43 2022] DELETE /identity/v3/projects/9d2e4b0153ff4563b91a906926526018 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:44.146565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f7f46-3802-4f84-b2d8-b01bf701fcbb None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:44.146844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f7f46-3802-4f84-b2d8-b01bf701fcbb None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:44.147109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f7f46-3802-4f84-b2d8-b01bf701fcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:44.168128 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-8d5f7f46-3802-4f84-b2d8-b01bf701fcbb None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:44.168840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8d5f7f46-3802-4f84-b2d8-b01bf701fcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:44.228135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 985/1981] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:44.236041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3157c0-f4ed-43d2-8478-9672ecd90e33 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:44.236265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3157c0-f4ed-43d2-8478-9672ecd90e33 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:44.236602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3157c0-f4ed-43d2-8478-9672ecd90e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:44.258437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-ab3157c0-f4ed-43d2-8478-9672ecd90e33 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:44.259690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ab3157c0-f4ed-43d2-8478-9672ecd90e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:44.268478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 997/1982] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:44.272709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f7f46-3802-4f84-b2d8-b01bf701fcbb None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:44.282808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f7f46-3802-4f84-b2d8-b01bf701fcbb None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:44.288716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4c9b7-1e0d-41cb-8e4c-45a4af298259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:44.289308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4c9b7-1e0d-41cb-8e4c-45a4af298259 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:44.289579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4c9b7-1e0d-41cb-8e4c-45a4af298259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:44.289865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4c9b7-1e0d-41cb-8e4c-45a4af298259 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:44.290503 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4c9b7-1e0d-41cb-8e4c-45a4af298259 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:43:44.290797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4c9b7-1e0d-41cb-8e4c-45a4af298259 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:44.292360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4c9b7-1e0d-41cb-8e4c-45a4af298259 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:44.305241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 986/1983] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:43:44.309642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3157c0-f4ed-43d2-8478-9672ecd90e33 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:44.319629 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3157c0-f4ed-43d2-8478-9672ecd90e33 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:44.324290 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d351ee-bf99-4883-a81c-0bb9dd4d847f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:44.324876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d351ee-bf99-4883-a81c-0bb9dd4d847f admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:44.325146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d351ee-bf99-4883-a81c-0bb9dd4d847f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:44.325410 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d351ee-bf99-4883-a81c-0bb9dd4d847f admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:44.326092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d351ee-bf99-4883-a81c-0bb9dd4d847f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:44.327963 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d351ee-bf99-4883-a81c-0bb9dd4d847f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:44.434984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 998/1984] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:44 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:44.442884 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4c9b7-1e0d-41cb-8e4c-45a4af298259 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:44.453168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4c9b7-1e0d-41cb-8e4c-45a4af298259 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:44.512709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc8300-9a1e-414c-ab01-7200620a2215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:44.513444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc8300-9a1e-414c-ab01-7200620a2215 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:44.513729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc8300-9a1e-414c-ab01-7200620a2215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:44.514028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc8300-9a1e-414c-ab01-7200620a2215 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:44.514817 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc8300-9a1e-414c-ab01-7200620a2215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:44.516727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc8300-9a1e-414c-ab01-7200620a2215 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:44.619605 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 987/1985] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:44 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:43:44.623765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d351ee-bf99-4883-a81c-0bb9dd4d847f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:44.634517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d351ee-bf99-4883-a81c-0bb9dd4d847f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:44.639223 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363f1c0-2026-430c-b248-6f755dddebff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:44.639902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363f1c0-2026-430c-b248-6f755dddebff admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:44.640173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363f1c0-2026-430c-b248-6f755dddebff admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:44.640435 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363f1c0-2026-430c-b248-6f755dddebff admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:44.641068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363f1c0-2026-430c-b248-6f755dddebff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:44.642669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363f1c0-2026-430c-b248-6f755dddebff admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:44.653516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 999/1986] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:44.657308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc8300-9a1e-414c-ab01-7200620a2215 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:44.667376 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc8300-9a1e-414c-ab01-7200620a2215 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:44.672059 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c458f5-001d-4980-b24f-662ef24dee1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:44.672644 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c458f5-001d-4980-b24f-662ef24dee1e admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:44.672918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c458f5-001d-4980-b24f-662ef24dee1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:44.673184 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c458f5-001d-4980-b24f-662ef24dee1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:44.673813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c458f5-001d-4980-b24f-662ef24dee1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:44.675623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c458f5-001d-4980-b24f-662ef24dee1e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:44.686004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 988/1987] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:44.690452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363f1c0-2026-430c-b248-6f755dddebff admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:44.700067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363f1c0-2026-430c-b248-6f755dddebff admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:44.704573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8d74f-cb2f-42bd-98a3-ae4c1db02399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:44.705216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8d74f-cb2f-42bd-98a3-ae4c1db02399 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:44.706150 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8d74f-cb2f-42bd-98a3-ae4c1db02399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:44.706439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8d74f-cb2f-42bd-98a3-ae4c1db02399 admin admin] PATH_INFO: `/v3/projects/6c60f20274994638a50f21b037541c12/users/084da259acad41a6b824d1aa7bc9f580/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:44.722020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8d74f-cb2f-42bd-98a3-ae4c1db02399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c60f20274994638a50f21b037541c12, user_id=084da259acad41a6b824d1aa7bc9f580, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:44.724630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8d74f-cb2f-42bd-98a3-ae4c1db02399 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:44.790909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1000/1988] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:44 2022] PUT /identity/v3/projects/6c60f20274994638a50f21b037541c12/users/084da259acad41a6b824d1aa7bc9f580/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:44.801321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd2e95-a2ee-4ab2-a7bf-3766d3d30b1d None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:44.801602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd2e95-a2ee-4ab2-a7bf-3766d3d30b1d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:44.801897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd2e95-a2ee-4ab2-a7bf-3766d3d30b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:44.829336 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-bbfd2e95-a2ee-4ab2-a7bf-3766d3d30b1d None None] MFA Rules not processed for user `084da259acad41a6b824d1aa7bc9f580`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:44.831235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-bbfd2e95-a2ee-4ab2-a7bf-3766d3d30b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:44.840148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:44.895137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 989/1989] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:44.906399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8d74f-cb2f-42bd-98a3-ae4c1db02399 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:44.908083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7af8d74f-cb2f-42bd-98a3-ae4c1db02399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:44.964807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8d74f-cb2f-42bd-98a3-ae4c1db02399 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:44.968465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4c814-5898-4cc1-86f1-0201ddfac742 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d96ec6235843c5b67d87a3d0ff0f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:44.969061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4c814-5898-4cc1-86f1-0201ddfac742 service glance] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:44.969329 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4c814-5898-4cc1-86f1-0201ddfac742 service glance] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:44.969594 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4c814-5898-4cc1-86f1-0201ddfac742 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:44.978003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4c814-5898-4cc1-86f1-0201ddfac742 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:44.979406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4c814-5898-4cc1-86f1-0201ddfac742 service glance] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:44.989060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1001/1990] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:43:46.894924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd2e95-a2ee-4ab2-a7bf-3766d3d30b1d None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:46.897179 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-bbfd2e95-a2ee-4ab2-a7bf-3766d3d30b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:46.962006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd2e95-a2ee-4ab2-a7bf-3766d3d30b1d None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:46.966492 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:46.967113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:46.967387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:46.967652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] PATH_INFO: `/v3/users/662fa45e9ffd48eda58470ceeac3866d` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:46.968764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] RBAC: Authorizing `identity:delete_user(user_id=662fa45e9ffd48eda58470ceeac3866d)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:46.970775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:47.131075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '662fa45e9ffd48eda58470ceeac3866d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:47.131722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '662fa45e9ffd48eda58470ceeac3866d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:47.184927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '662fa45e9ffd48eda58470ceeac3866d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:47.192436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '662fa45e9ffd48eda58470ceeac3866d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:47.193076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '662fa45e9ffd48eda58470ceeac3866d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:47.211923 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 990/1991] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:46 2022] DELETE /identity/v3/users/662fa45e9ffd48eda58470ceeac3866d => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:47.216241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa335b24-7f72-4edc-8cf4-4c7233391591 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:47.216696 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa335b24-7f72-4edc-8cf4-4c7233391591 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:47.216786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa335b24-7f72-4edc-8cf4-4c7233391591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:47.236748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-aa335b24-7f72-4edc-8cf4-4c7233391591 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:47.238120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-aa335b24-7f72-4edc-8cf4-4c7233391591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:47.314289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1002/1992] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:47.325292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:47.328229 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:47.369804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:47.395198 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0159a97f-274a-419d-9a5d-23091e3bb94b admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:47.398968 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18319b93-beb6-4310-919b-271a22c86c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:47.399578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18319b93-beb6-4310-919b-271a22c86c3f service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:47.399868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18319b93-beb6-4310-919b-271a22c86c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:47.400159 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18319b93-beb6-4310-919b-271a22c86c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:47.410512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18319b93-beb6-4310-919b-271a22c86c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:47.412338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18319b93-beb6-4310-919b-271a22c86c3f service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:47.423960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 991/1993] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:43:47.903252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa335b24-7f72-4edc-8cf4-4c7233391591 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:47.905380 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-aa335b24-7f72-4edc-8cf4-4c7233391591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:47.926417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa335b24-7f72-4edc-8cf4-4c7233391591 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:48.057620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:48.057620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:48.057620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:48.057620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] PATH_INFO: `/v3/projects/035a55a6a0684843b995b2bb854f5f99` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:48.060744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=035a55a6a0684843b995b2bb854f5f99)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:48.060744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:48.060744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:48.136966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '035a55a6a0684843b995b2bb854f5f99', 'request_id': 'req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:48.143260 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '035a55a6a0684843b995b2bb854f5f99', 'request_id': 'req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:48.150318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '035a55a6a0684843b995b2bb854f5f99', 'request_id': 'req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:48.225187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] The token cache is being invalidate because project 035a55a6a0684843b995b2bb854f5f99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:48.225891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:48.226197 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:48.227923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1003/1994] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:47 2022] DELETE /identity/v3/projects/035a55a6a0684843b995b2bb854f5f99 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:48.232239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18319b93-beb6-4310-919b-271a22c86c3f service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:48.242463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18319b93-beb6-4310-919b-271a22c86c3f service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:48.301935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591b501-920f-44c2-a825-a791230977a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:48.302569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591b501-920f-44c2-a825-a791230977a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:48.302843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591b501-920f-44c2-a825-a791230977a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:48.303110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591b501-920f-44c2-a825-a791230977a6 admin admin] PATH_INFO: `/v3/users/b07f4bcd99284e4babc1d80d27c04d99` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:48.304345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591b501-920f-44c2-a825-a791230977a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b07f4bcd99284e4babc1d80d27c04d99)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:48.306280 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591b501-920f-44c2-a825-a791230977a6 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:48.504612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-1591b501-920f-44c2-a825-a791230977a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b07f4bcd99284e4babc1d80d27c04d99'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:48.505402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-1591b501-920f-44c2-a825-a791230977a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b07f4bcd99284e4babc1d80d27c04d99'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:48.596050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-1591b501-920f-44c2-a825-a791230977a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b07f4bcd99284e4babc1d80d27c04d99'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:48.603745 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-1591b501-920f-44c2-a825-a791230977a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b07f4bcd99284e4babc1d80d27c04d99'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:48.604390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-1591b501-920f-44c2-a825-a791230977a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b07f4bcd99284e4babc1d80d27c04d99'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:48.622440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 992/1995] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:48 2022] DELETE /identity/v3/users/b07f4bcd99284e4babc1d80d27c04d99 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:49.403552 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:49.405007 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:49.426821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4c155a-681c-44bd-bd9c-c03b037ad3a5 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:49.515493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:49.521973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:49.521973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:49.521973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157 admin admin] PATH_INFO: `/v3/projects/02b0007126c044ed9663c8d19cca92a7` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:49.521973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02b0007126c044ed9663c8d19cca92a7)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:49.521973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:49.662706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02b0007126c044ed9663c8d19cca92a7', 'request_id': 'req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:49.671792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02b0007126c044ed9663c8d19cca92a7', 'request_id': 'req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:49.678953 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02b0007126c044ed9663c8d19cca92a7', 'request_id': 'req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:49.778001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157 admin admin] The token cache is being invalidate because project 02b0007126c044ed9663c8d19cca92a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:49.778001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:49.778001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-d81bae9d-4ddc-4efd-8b8c-1aa8bb59d157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:49.778001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1004/1996] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:49 2022] DELETE /identity/v3/projects/02b0007126c044ed9663c8d19cca92a7 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:49.781642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ad5db-a762-40e5-bf7c-8f9520d3328a None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:49.782151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ad5db-a762-40e5-bf7c-8f9520d3328a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:49.782252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ad5db-a762-40e5-bf7c-8f9520d3328a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:49.802901 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-694ad5db-a762-40e5-bf7c-8f9520d3328a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:49.806747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-694ad5db-a762-40e5-bf7c-8f9520d3328a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:49.855844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:49.877202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bdb03-ace5-4636-b706-bd199c7017c1 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:49.877489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bdb03-ace5-4636-b706-bd199c7017c1 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:49.877759 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5bdb03-ace5-4636-b706-bd199c7017c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:49.880736 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 993/1997] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:49.904205 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-cb5bdb03-ace5-4636-b706-bd199c7017c1 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:49.905425 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-cb5bdb03-ace5-4636-b706-bd199c7017c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:49.919803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ad5db-a762-40e5-bf7c-8f9520d3328a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:49.926341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1005/1998] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:49.931450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ad5db-a762-40e5-bf7c-8f9520d3328a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:49.936627 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3ad42-78f7-4cd4-a6db-5dfa6f1ee592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:49.937337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3ad42-78f7-4cd4-a6db-5dfa6f1ee592 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:49.937807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3ad42-78f7-4cd4-a6db-5dfa6f1ee592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:49.938147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3ad42-78f7-4cd4-a6db-5dfa6f1ee592 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:49.938775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3ad42-78f7-4cd4-a6db-5dfa6f1ee592 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:43:49.939092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3ad42-78f7-4cd4-a6db-5dfa6f1ee592 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:49.940701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3ad42-78f7-4cd4-a6db-5dfa6f1ee592 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:49.957642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 994/1999] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:43:49.958816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5bdb03-ace5-4636-b706-bd199c7017c1 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:49.968212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5bdb03-ace5-4636-b706-bd199c7017c1 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:49.974382 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b41c9-61d0-4b18-869a-4cc6f4374a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:49.975030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b41c9-61d0-4b18-869a-4cc6f4374a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:49.975305 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b41c9-61d0-4b18-869a-4cc6f4374a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:49.975574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b41c9-61d0-4b18-869a-4cc6f4374a0e admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:49.976200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b41c9-61d0-4b18-869a-4cc6f4374a0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:49.978124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b41c9-61d0-4b18-869a-4cc6f4374a0e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:50.081120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1006/2000] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:49 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:50.085147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3ad42-78f7-4cd4-a6db-5dfa6f1ee592 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:50.110049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3ad42-78f7-4cd4-a6db-5dfa6f1ee592 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:50.182880 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a2ad1-2e94-4a9e-9343-2fabdafa2da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:50.190743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a2ad1-2e94-4a9e-9343-2fabdafa2da5 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:50.190743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a2ad1-2e94-4a9e-9343-2fabdafa2da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:50.190743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a2ad1-2e94-4a9e-9343-2fabdafa2da5 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:50.190743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a2ad1-2e94-4a9e-9343-2fabdafa2da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:50.190743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a2ad1-2e94-4a9e-9343-2fabdafa2da5 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:50.276601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 995/2001] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:50 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:50.280996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b41c9-61d0-4b18-869a-4cc6f4374a0e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:50.291496 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b41c9-61d0-4b18-869a-4cc6f4374a0e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:50.297234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd426ca9-10f8-4ec8-9e0c-c845671c55c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:50.297927 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd426ca9-10f8-4ec8-9e0c-c845671c55c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:50.299644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd426ca9-10f8-4ec8-9e0c-c845671c55c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:50.299644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd426ca9-10f8-4ec8-9e0c-c845671c55c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:50.299644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd426ca9-10f8-4ec8-9e0c-c845671c55c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:50.302368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd426ca9-10f8-4ec8-9e0c-c845671c55c7 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:50.304034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:50.317355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1007/2002] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:50.322694 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a2ad1-2e94-4a9e-9343-2fabdafa2da5 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:50.333804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a2ad1-2e94-4a9e-9343-2fabdafa2da5 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:50.338726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00926e4-5de1-4982-bc9d-a540b31231fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:50.339336 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00926e4-5de1-4982-bc9d-a540b31231fe admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:50.339903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00926e4-5de1-4982-bc9d-a540b31231fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:50.339903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00926e4-5de1-4982-bc9d-a540b31231fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:50.340621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00926e4-5de1-4982-bc9d-a540b31231fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:50.342232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00926e4-5de1-4982-bc9d-a540b31231fe admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:50.353745 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 996/2003] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:50.357951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd426ca9-10f8-4ec8-9e0c-c845671c55c7 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:50.368957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd426ca9-10f8-4ec8-9e0c-c845671c55c7 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:50.373732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b988f8-0e79-4806-8762-3b61a15c8feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:50.374416 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b988f8-0e79-4806-8762-3b61a15c8feb admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:50.374697 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b988f8-0e79-4806-8762-3b61a15c8feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:50.374966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b988f8-0e79-4806-8762-3b61a15c8feb admin admin] PATH_INFO: `/v3/projects/410c3bb930d74d02807b0db4ab32c50e/users/62e3850e3d554f3f94d9afeedfd26d7d/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:50.391887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b988f8-0e79-4806-8762-3b61a15c8feb admin admin] RBAC: Authorizing `identity:create_grant(project_id=410c3bb930d74d02807b0db4ab32c50e, user_id=62e3850e3d554f3f94d9afeedfd26d7d, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:50.395450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b988f8-0e79-4806-8762-3b61a15c8feb admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:50.464776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1008/2004] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:50 2022] PUT /identity/v3/projects/410c3bb930d74d02807b0db4ab32c50e/users/62e3850e3d554f3f94d9afeedfd26d7d/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:50.482622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6bbba-cd96-4438-a330-a0286c082c20 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:50.482942 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6bbba-cd96-4438-a330-a0286c082c20 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:50.483272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6bbba-cd96-4438-a330-a0286c082c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:50.511855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-bce6bbba-cd96-4438-a330-a0286c082c20 None None] MFA Rules not processed for user `62e3850e3d554f3f94d9afeedfd26d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:50.513076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-bce6bbba-cd96-4438-a330-a0286c082c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:50.578252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 997/2005] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:50.582791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b988f8-0e79-4806-8762-3b61a15c8feb admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:50.648326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b988f8-0e79-4806-8762-3b61a15c8feb admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:50.653587 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd027a-a06b-4c57-b556-f99125968ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:50.654230 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd027a-a06b-4c57-b556-f99125968ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:50.654502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd027a-a06b-4c57-b556-f99125968ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:50.654769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd027a-a06b-4c57-b556-f99125968ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:50.655403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd027a-a06b-4c57-b556-f99125968ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:50.657298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd027a-a06b-4c57-b556-f99125968ea6 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:50.795481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1009/2006] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:50 2022] POST /identity/v3/projects => generated 385 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:50.796527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6bbba-cd96-4438-a330-a0286c082c20 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:50.808163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6bbba-cd96-4438-a330-a0286c082c20 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:50.860219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:50.872275 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4242d7-b117-459f-9a16-86207068419f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:50.872877 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4242d7-b117-459f-9a16-86207068419f admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:50.873146 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4242d7-b117-459f-9a16-86207068419f admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:50.873408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4242d7-b117-459f-9a16-86207068419f admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:50.874103 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4242d7-b117-459f-9a16-86207068419f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:50.876014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4242d7-b117-459f-9a16-86207068419f admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:50.962796 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 998/2007] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:50 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:43:50.966905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd027a-a06b-4c57-b556-f99125968ea6 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:50.978066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd027a-a06b-4c57-b556-f99125968ea6 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:50.983067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaeba6a-eb7a-47a8-a8ae-7dbe3a5c97be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:50.986881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaeba6a-eb7a-47a8-a8ae-7dbe3a5c97be admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:50.986881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaeba6a-eb7a-47a8-a8ae-7dbe3a5c97be admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:50.986881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaeba6a-eb7a-47a8-a8ae-7dbe3a5c97be admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:50.986881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaeba6a-eb7a-47a8-a8ae-7dbe3a5c97be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:50.986881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaeba6a-eb7a-47a8-a8ae-7dbe3a5c97be admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:50.998173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1010/2008] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:51.002803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4242d7-b117-459f-9a16-86207068419f admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:51.006186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-db4242d7-b117-459f-9a16-86207068419f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:51.008590 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaeba6a-eb7a-47a8-a8ae-7dbe3a5c97be admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:51.018415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaeba6a-eb7a-47a8-a8ae-7dbe3a5c97be admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:51.027860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4242d7-b117-459f-9a16-86207068419f admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:51.027860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531d940-343c-40f3-a767-098d187bd5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:51.028390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f37b4a-d14f-4e43-8105-721d52047ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:51.028539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531d940-343c-40f3-a767-098d187bd5c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:51.028539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531d940-343c-40f3-a767-098d187bd5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:51.028702 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f37b4a-d14f-4e43-8105-721d52047ec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:51.028799 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531d940-343c-40f3-a767-098d187bd5c0 admin admin] PATH_INFO: `/v3/users/6beb52c7292d491cbb60f5474beb7f95` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:51.028885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f37b4a-d14f-4e43-8105-721d52047ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:51.028885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f37b4a-d14f-4e43-8105-721d52047ec6 admin admin] PATH_INFO: `/v3/projects/1e8c1ae7c0dd4dd383327d4b94a5b9ac/users/93bca6ca031c4cd3bfb3aa792ed4ba02/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:51.029046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531d940-343c-40f3-a767-098d187bd5c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6beb52c7292d491cbb60f5474beb7f95)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:51.029046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531d940-343c-40f3-a767-098d187bd5c0 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:51.042133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f37b4a-d14f-4e43-8105-721d52047ec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8c1ae7c0dd4dd383327d4b94a5b9ac, user_id=93bca6ca031c4cd3bfb3aa792ed4ba02, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:51.044763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f37b4a-d14f-4e43-8105-721d52047ec6 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:51.133589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1011/2009] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:51 2022] PUT /identity/v3/projects/1e8c1ae7c0dd4dd383327d4b94a5b9ac/users/93bca6ca031c4cd3bfb3aa792ed4ba02/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:51.137535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f37b4a-d14f-4e43-8105-721d52047ec6 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:51.219670 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f37b4a-d14f-4e43-8105-721d52047ec6 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:51.227438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d65ec-4ba8-4c4e-95de-5a82583fe0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:51.228158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d65ec-4ba8-4c4e-95de-5a82583fe0d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:51.228442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d65ec-4ba8-4c4e-95de-5a82583fe0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:51.228710 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d65ec-4ba8-4c4e-95de-5a82583fe0d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:51.229390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d65ec-4ba8-4c4e-95de-5a82583fe0d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:51.231451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d65ec-4ba8-4c4e-95de-5a82583fe0d9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:51.248181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1012/2010] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:43:51.254568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d65ec-4ba8-4c4e-95de-5a82583fe0d9 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:51.266298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d65ec-4ba8-4c4e-95de-5a82583fe0d9 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:51.273014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937e5de-4a61-4880-bd35-eb30f91861ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:51.276421 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937e5de-4a61-4880-bd35-eb30f91861ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:51.276830 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937e5de-4a61-4880-bd35-eb30f91861ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:51.280472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937e5de-4a61-4880-bd35-eb30f91861ae admin admin] PATH_INFO: `/v3/projects/1e8c1ae7c0dd4dd383327d4b94a5b9ac/users/93bca6ca031c4cd3bfb3aa792ed4ba02/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:51.280728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c531d940-343c-40f3-a767-098d187bd5c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6beb52c7292d491cbb60f5474beb7f95'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:51.281397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c531d940-343c-40f3-a767-098d187bd5c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6beb52c7292d491cbb60f5474beb7f95'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:51.282513 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937e5de-4a61-4880-bd35-eb30f91861ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8c1ae7c0dd4dd383327d4b94a5b9ac, user_id=93bca6ca031c4cd3bfb3aa792ed4ba02, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:51.287303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937e5de-4a61-4880-bd35-eb30f91861ae admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:51.299250 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1013/2011] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:51 2022] PUT /identity/v3/projects/1e8c1ae7c0dd4dd383327d4b94a5b9ac/users/93bca6ca031c4cd3bfb3aa792ed4ba02/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:51.310115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91c906-cf17-404a-ba7c-6eff8ce7f27f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:51.314617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91c906-cf17-404a-ba7c-6eff8ce7f27f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:51.314617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac91c906-cf17-404a-ba7c-6eff8ce7f27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:51.320790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:51.340585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-ac91c906-cf17-404a-ba7c-6eff8ce7f27f None None] MFA Rules not processed for user `93bca6ca031c4cd3bfb3aa792ed4ba02`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:51.341811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ac91c906-cf17-404a-ba7c-6eff8ce7f27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:51.381285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c531d940-343c-40f3-a767-098d187bd5c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6beb52c7292d491cbb60f5474beb7f95'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:51.387566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c531d940-343c-40f3-a767-098d187bd5c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6beb52c7292d491cbb60f5474beb7f95'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:51.388915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c531d940-343c-40f3-a767-098d187bd5c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6beb52c7292d491cbb60f5474beb7f95'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:51.408237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 999/2012] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:51 2022] DELETE /identity/v3/users/6beb52c7292d491cbb60f5474beb7f95 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:51.413933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da02f7-a15d-41ab-9233-035e269cff89 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:51.413933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da02f7-a15d-41ab-9233-035e269cff89 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:51.414302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3da02f7-a15d-41ab-9233-035e269cff89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:51.418460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1014/2013] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:51.430126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91c906-cf17-404a-ba7c-6eff8ce7f27f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:51.431475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ac91c906-cf17-404a-ba7c-6eff8ce7f27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:51.433870 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c3da02f7-a15d-41ab-9233-035e269cff89 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:51.435126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c3da02f7-a15d-41ab-9233-035e269cff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:51.496427 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1000/2014] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:51.496850 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac91c906-cf17-404a-ba7c-6eff8ce7f27f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:51.508700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da02f7-a15d-41ab-9233-035e269cff89 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:51.512160 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c3da02f7-a15d-41ab-9233-035e269cff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:51.558534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa4827-75af-48ac-807e-7cbd22486b45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:51.559147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa4827-75af-48ac-807e-7cbd22486b45 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:51.559510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa4827-75af-48ac-807e-7cbd22486b45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:51.559694 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa4827-75af-48ac-807e-7cbd22486b45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:51.561456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-1ffa4827-75af-48ac-807e-7cbd22486b45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:51.585970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3da02f7-a15d-41ab-9233-035e269cff89 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:51.591488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d9543-4255-4725-a5df-c91a819f9047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:51.592738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d9543-4255-4725-a5df-c91a819f9047 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:51.593232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d9543-4255-4725-a5df-c91a819f9047 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:51.593503 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d9543-4255-4725-a5df-c91a819f9047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:51.601765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:51.606653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d9543-4255-4725-a5df-c91a819f9047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:51.608090 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d9543-4255-4725-a5df-c91a819f9047 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:51.618393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1001/2015] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:43:51.626774 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa4827-75af-48ac-807e-7cbd22486b45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:51.628366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa4827-75af-48ac-807e-7cbd22486b45 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:51.633883 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:51.637685 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1015/2016] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:43:52.118801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d9543-4255-4725-a5df-c91a819f9047 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:52.120990 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-582d9543-4255-4725-a5df-c91a819f9047 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:52.136011 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d9543-4255-4725-a5df-c91a819f9047 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:52.141995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:52.142471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:52.142754 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:52.143028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] PATH_INFO: `/v3/projects/bf8299edcc474c43b7613d1d2c915c4a` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:52.144442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf8299edcc474c43b7613d1d2c915c4a)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:52.149630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:52.311746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf8299edcc474c43b7613d1d2c915c4a', 'request_id': 'req-4a50351e-2da9-4a82-9477-427e0e7fad44'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:52.318098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf8299edcc474c43b7613d1d2c915c4a', 'request_id': 'req-4a50351e-2da9-4a82-9477-427e0e7fad44'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:52.324639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf8299edcc474c43b7613d1d2c915c4a', 'request_id': 'req-4a50351e-2da9-4a82-9477-427e0e7fad44'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:52.407385 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] The token cache is being invalidate because project bf8299edcc474c43b7613d1d2c915c4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:52.407707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:52.408333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:52.409959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1002/2017] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:52 2022] DELETE /identity/v3/projects/bf8299edcc474c43b7613d1d2c915c4a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:52.414088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa4827-75af-48ac-807e-7cbd22486b45 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:52.424957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa4827-75af-48ac-807e-7cbd22486b45 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:52.494118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b65b78-76a0-44a1-89d6-d263164a17ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:52.494911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b65b78-76a0-44a1-89d6-d263164a17ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:52.495186 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b65b78-76a0-44a1-89d6-d263164a17ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:52.495450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b65b78-76a0-44a1-89d6-d263164a17ad admin admin] PATH_INFO: `/v3/users/132824f9284b471cac6cb850815529c1` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:52.496566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b65b78-76a0-44a1-89d6-d263164a17ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=132824f9284b471cac6cb850815529c1)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:52.498566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b65b78-76a0-44a1-89d6-d263164a17ad admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:52.658792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c2b65b78-76a0-44a1-89d6-d263164a17ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '132824f9284b471cac6cb850815529c1'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:52.664860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c2b65b78-76a0-44a1-89d6-d263164a17ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '132824f9284b471cac6cb850815529c1'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:52.710554 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c2b65b78-76a0-44a1-89d6-d263164a17ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '132824f9284b471cac6cb850815529c1'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:52.711181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c2b65b78-76a0-44a1-89d6-d263164a17ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '132824f9284b471cac6cb850815529c1'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:52.719095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c2b65b78-76a0-44a1-89d6-d263164a17ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '132824f9284b471cac6cb850815529c1'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:52.743070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1016/2018] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:43:52 2022] DELETE /identity/v3/users/132824f9284b471cac6cb850815529c1 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:53.194984 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:53.196485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:53.209746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50351e-2da9-4a82-9477-427e0e7fad44 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:53.271698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b73b5-4202-4891-beed-3db4295955c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:53.272260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b73b5-4202-4891-beed-3db4295955c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:53.272532 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b73b5-4202-4891-beed-3db4295955c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:53.272800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b73b5-4202-4891-beed-3db4295955c1 admin admin] PATH_INFO: `/v3/projects/8a61676df1fe41cab96bbe3fce56af35` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:53.273969 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b73b5-4202-4891-beed-3db4295955c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a61676df1fe41cab96bbe3fce56af35)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:53.277579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b73b5-4202-4891-beed-3db4295955c1 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:53.361581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-565b73b5-4202-4891-beed-3db4295955c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a61676df1fe41cab96bbe3fce56af35', 'request_id': 'req-565b73b5-4202-4891-beed-3db4295955c1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:53.367595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-565b73b5-4202-4891-beed-3db4295955c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a61676df1fe41cab96bbe3fce56af35', 'request_id': 'req-565b73b5-4202-4891-beed-3db4295955c1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:53.373632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-565b73b5-4202-4891-beed-3db4295955c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a61676df1fe41cab96bbe3fce56af35', 'request_id': 'req-565b73b5-4202-4891-beed-3db4295955c1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:53.423461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-565b73b5-4202-4891-beed-3db4295955c1 admin admin] The token cache is being invalidate because project 8a61676df1fe41cab96bbe3fce56af35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:43:53.423744 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-565b73b5-4202-4891-beed-3db4295955c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:53.424443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-565b73b5-4202-4891-beed-3db4295955c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:43:53.426845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1003/2019] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:43:53 2022] DELETE /identity/v3/projects/8a61676df1fe41cab96bbe3fce56af35 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:43:53.431729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed724e91-546b-4690-8199-19bb45a6b74a None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:53.432013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed724e91-546b-4690-8199-19bb45a6b74a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:53.432489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed724e91-546b-4690-8199-19bb45a6b74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:53.452562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-ed724e91-546b-4690-8199-19bb45a6b74a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:53.453776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ed724e91-546b-4690-8199-19bb45a6b74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:53.514308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1017/2020] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:43:53.523435 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47d578-f963-49ad-835c-421b718d45a6 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:53.523653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47d578-f963-49ad-835c-421b718d45a6 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:53.523928 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47d578-f963-49ad-835c-421b718d45a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:53.539359 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:53.545794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-ca47d578-f963-49ad-835c-421b718d45a6 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:53.547099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-ca47d578-f963-49ad-835c-421b718d45a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:53.555989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1004/2021] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:53.560240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed724e91-546b-4690-8199-19bb45a6b74a None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:53.570855 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed724e91-546b-4690-8199-19bb45a6b74a None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:53.575595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d435f-d084-472d-90c9-50190a8f4b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:53.576282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d435f-d084-472d-90c9-50190a8f4b4d admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:53.576622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d435f-d084-472d-90c9-50190a8f4b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:53.576964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d435f-d084-472d-90c9-50190a8f4b4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:53.577676 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d435f-d084-472d-90c9-50190a8f4b4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:43:53.579437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d435f-d084-472d-90c9-50190a8f4b4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:53.580091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d435f-d084-472d-90c9-50190a8f4b4d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:53.596323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1018/2022] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:43:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:43:53.598709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47d578-f963-49ad-835c-421b718d45a6 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:53.608550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47d578-f963-49ad-835c-421b718d45a6 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:53.614362 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5125bfc-385f-46a9-a742-121fba7c8d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:53.615180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5125bfc-385f-46a9-a742-121fba7c8d57 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:53.615517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5125bfc-385f-46a9-a742-121fba7c8d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:53.615857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5125bfc-385f-46a9-a742-121fba7c8d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:53.616618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5125bfc-385f-46a9-a742-121fba7c8d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:53.618908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5125bfc-385f-46a9-a742-121fba7c8d57 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:53.692964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1005/2023] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:43:53 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:53.694253 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d435f-d084-472d-90c9-50190a8f4b4d admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:53.704695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d435f-d084-472d-90c9-50190a8f4b4d admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:53.708580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:43:53.762543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4614442-55e1-4151-9eb7-4148a41db913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:53.763158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4614442-55e1-4151-9eb7-4148a41db913 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:53.763598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4614442-55e1-4151-9eb7-4148a41db913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:53.763855 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4614442-55e1-4151-9eb7-4148a41db913 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:53.764591 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4614442-55e1-4151-9eb7-4148a41db913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:53.766543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4614442-55e1-4151-9eb7-4148a41db913 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:53.887763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1019/2024] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:43:53 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:43:53.891917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5125bfc-385f-46a9-a742-121fba7c8d57 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:53.902300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5125bfc-385f-46a9-a742-121fba7c8d57 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:53.907511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c24335-e604-438b-a37e-f4a8d379c6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:53.915264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c24335-e604-438b-a37e-f4a8d379c6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:53.915264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c24335-e604-438b-a37e-f4a8d379c6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:53.915264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c24335-e604-438b-a37e-f4a8d379c6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:53.915264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c24335-e604-438b-a37e-f4a8d379c6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:53.915264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c24335-e604-438b-a37e-f4a8d379c6a0 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:53.921398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1006/2025] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:53.926004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4614442-55e1-4151-9eb7-4148a41db913 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:53.936233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4614442-55e1-4151-9eb7-4148a41db913 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:53.941534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474f626-a098-4055-a6b1-aed46761d718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:53.942354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f626-a098-4055-a6b1-aed46761d718 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:53.943822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f626-a098-4055-a6b1-aed46761d718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:53.943822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f626-a098-4055-a6b1-aed46761d718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:53.943822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474f626-a098-4055-a6b1-aed46761d718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:53.945185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474f626-a098-4055-a6b1-aed46761d718 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:53.957170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1020/2026] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:43:53.961365 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c24335-e604-438b-a37e-f4a8d379c6a0 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:53.971121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c24335-e604-438b-a37e-f4a8d379c6a0 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:53.975637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40be48-951f-4d23-9bf4-ab7543d30f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:53.976272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40be48-951f-4d23-9bf4-ab7543d30f2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:53.976542 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40be48-951f-4d23-9bf4-ab7543d30f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:53.976807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df40be48-951f-4d23-9bf4-ab7543d30f2d admin admin] PATH_INFO: `/v3/projects/c09db6f287454bd08071acbab824eee3/users/7ad59abaec3443578372bef1825cd36d/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:53.992050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40be48-951f-4d23-9bf4-ab7543d30f2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c09db6f287454bd08071acbab824eee3, user_id=7ad59abaec3443578372bef1825cd36d, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:53.994906 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df40be48-951f-4d23-9bf4-ab7543d30f2d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:54.075584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1007/2027] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:43:53 2022] PUT /identity/v3/projects/c09db6f287454bd08071acbab824eee3/users/7ad59abaec3443578372bef1825cd36d/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:43:54.087390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37876a83-f0c1-4f1c-82c9-1b7e6974e6df None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:54.087672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37876a83-f0c1-4f1c-82c9-1b7e6974e6df None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:54.087944 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37876a83-f0c1-4f1c-82c9-1b7e6974e6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:54.116118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-37876a83-f0c1-4f1c-82c9-1b7e6974e6df None None] MFA Rules not processed for user `7ad59abaec3443578372bef1825cd36d`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:43:54.117327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-37876a83-f0c1-4f1c-82c9-1b7e6974e6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:54.182510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1021/2028] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:43:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:43:54.194451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40be48-951f-4d23-9bf4-ab7543d30f2d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:43:54.196135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-df40be48-951f-4d23-9bf4-ab7543d30f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:43:54.253978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df40be48-951f-4d23-9bf4-ab7543d30f2d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:43:54.257645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca79dff-5baa-424b-a8e8-af4f5fdc235c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:43:54.258281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca79dff-5baa-424b-a8e8-af4f5fdc235c service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:43:54.258556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca79dff-5baa-424b-a8e8-af4f5fdc235c service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:43:54.258876 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca79dff-5baa-424b-a8e8-af4f5fdc235c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:43:54.267924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca79dff-5baa-424b-a8e8-af4f5fdc235c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:43:54.270056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca79dff-5baa-424b-a8e8-af4f5fdc235c service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:43:54.287698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1008/2029] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:43:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:04.179665 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:04.275157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:06.824535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37876a83-f0c1-4f1c-82c9-1b7e6974e6df None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:06.837396 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37876a83-f0c1-4f1c-82c9-1b7e6974e6df None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:06.841376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084fcb4-0845-4de1-9dd9-4f6d770e9b2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:06.842037 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084fcb4-0845-4de1-9dd9-4f6d770e9b2e service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:06.842317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084fcb4-0845-4de1-9dd9-4f6d770e9b2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:06.842584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084fcb4-0845-4de1-9dd9-4f6d770e9b2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:06.844380 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8084fcb4-0845-4de1-9dd9-4f6d770e9b2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:06.925279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084fcb4-0845-4de1-9dd9-4f6d770e9b2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:06.926820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084fcb4-0845-4de1-9dd9-4f6d770e9b2e service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:06.936940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1022/2030] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:08.246238 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca79dff-5baa-424b-a8e8-af4f5fdc235c service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:08.247767 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3ca79dff-5baa-424b-a8e8-af4f5fdc235c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:08.316474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca79dff-5baa-424b-a8e8-af4f5fdc235c service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:08.322316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:08.322944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:08.323216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:08.323481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] PATH_INFO: `/v3/users/68a9a2df341b4c01b458f16de90ae88b` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:08.324586 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68a9a2df341b4c01b458f16de90ae88b)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:08.326569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:08.559711 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68a9a2df341b4c01b458f16de90ae88b'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:08.560392 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68a9a2df341b4c01b458f16de90ae88b'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:08.624240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68a9a2df341b4c01b458f16de90ae88b'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:08.632507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68a9a2df341b4c01b458f16de90ae88b'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:08.633181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68a9a2df341b4c01b458f16de90ae88b'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:08.652340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1009/2031] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:08 2022] DELETE /identity/v3/users/68a9a2df341b4c01b458f16de90ae88b => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:08.662050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930dc8ea-2012-44a0-989a-de83e80a1232 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:08.662050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930dc8ea-2012-44a0-989a-de83e80a1232 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:08.662050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930dc8ea-2012-44a0-989a-de83e80a1232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:08.678514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-930dc8ea-2012-44a0-989a-de83e80a1232 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:08.679785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-930dc8ea-2012-44a0-989a-de83e80a1232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:08.737769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1023/2032] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:44:08.748093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:08.749512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:08.811927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a994cfc-bb68-401e-8bc1-5e304a794fa9 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:08.815461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce5f73-5c64-4976-aa17-136845a64a0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:08.816155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce5f73-5c64-4976-aa17-136845a64a0d service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:08.816446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce5f73-5c64-4976-aa17-136845a64a0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:08.816714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce5f73-5c64-4976-aa17-136845a64a0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:08.826109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce5f73-5c64-4976-aa17-136845a64a0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:08.827433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce5f73-5c64-4976-aa17-136845a64a0d service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:08.838046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1010/2033] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:09.268210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930dc8ea-2012-44a0-989a-de83e80a1232 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:09.269732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-930dc8ea-2012-44a0-989a-de83e80a1232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:09.289050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930dc8ea-2012-44a0-989a-de83e80a1232 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:09.294252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a8538-88fc-494f-b362-81cd3d37df14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:09.295130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a8538-88fc-494f-b362-81cd3d37df14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:09.295558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a8538-88fc-494f-b362-81cd3d37df14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:09.295977 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a8538-88fc-494f-b362-81cd3d37df14 admin admin] PATH_INFO: `/v3/projects/c1115ed4067b40f5a751fa75fc7fc295` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:09.309690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a8538-88fc-494f-b362-81cd3d37df14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1115ed4067b40f5a751fa75fc7fc295)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:09.313366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a8538-88fc-494f-b362-81cd3d37df14 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:09.450303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ca9a8538-88fc-494f-b362-81cd3d37df14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1115ed4067b40f5a751fa75fc7fc295', 'request_id': 'req-ca9a8538-88fc-494f-b362-81cd3d37df14'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:09.460178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ca9a8538-88fc-494f-b362-81cd3d37df14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1115ed4067b40f5a751fa75fc7fc295', 'request_id': 'req-ca9a8538-88fc-494f-b362-81cd3d37df14'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:09.468490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ca9a8538-88fc-494f-b362-81cd3d37df14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1115ed4067b40f5a751fa75fc7fc295', 'request_id': 'req-ca9a8538-88fc-494f-b362-81cd3d37df14'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:09.561494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ca9a8538-88fc-494f-b362-81cd3d37df14 admin admin] The token cache is being invalidate because project c1115ed4067b40f5a751fa75fc7fc295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:09.561808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ca9a8538-88fc-494f-b362-81cd3d37df14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:09.562520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ca9a8538-88fc-494f-b362-81cd3d37df14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:09.564291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1024/2034] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:09 2022] DELETE /identity/v3/projects/c1115ed4067b40f5a751fa75fc7fc295 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:09.569790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7e902-83ca-4418-8111-101f8d256f42 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:09.570311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7e902-83ca-4418-8111-101f8d256f42 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:09.570403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e7e902-83ca-4418-8111-101f8d256f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:09.591520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-07e7e902-83ca-4418-8111-101f8d256f42 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:09.592948 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-07e7e902-83ca-4418-8111-101f8d256f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:09.613701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:09.684646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1011/2035] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:09.692174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac690d0-82d6-413c-87a0-73d9e8eb6b3c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:09.692681 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac690d0-82d6-413c-87a0-73d9e8eb6b3c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:09.692914 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac690d0-82d6-413c-87a0-73d9e8eb6b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:09.720665 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-cac690d0-82d6-413c-87a0-73d9e8eb6b3c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:09.726660 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-cac690d0-82d6-413c-87a0-73d9e8eb6b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:09.738498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1025/2036] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:44:09.746396 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7e902-83ca-4418-8111-101f8d256f42 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:09.762730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e7e902-83ca-4418-8111-101f8d256f42 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:09.767874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94d491-d9ea-441d-8f6c-b170dbfc6152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:09.768552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94d491-d9ea-441d-8f6c-b170dbfc6152 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:09.768888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94d491-d9ea-441d-8f6c-b170dbfc6152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:09.769264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94d491-d9ea-441d-8f6c-b170dbfc6152 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:09.770073 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d94d491-d9ea-441d-8f6c-b170dbfc6152 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:44:09.770464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d94d491-d9ea-441d-8f6c-b170dbfc6152 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:09.772357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d94d491-d9ea-441d-8f6c-b170dbfc6152 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:09.786214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1012/2037] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:44:09.790821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac690d0-82d6-413c-87a0-73d9e8eb6b3c None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:09.804360 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac690d0-82d6-413c-87a0-73d9e8eb6b3c None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:09.810029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e4661-769c-48ae-b7ec-2d0e69ebe8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:09.810643 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e4661-769c-48ae-b7ec-2d0e69ebe8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:09.810918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e4661-769c-48ae-b7ec-2d0e69ebe8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:09.812419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e4661-769c-48ae-b7ec-2d0e69ebe8c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:09.812419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e4661-769c-48ae-b7ec-2d0e69ebe8c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:09.814615 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151e4661-769c-48ae-b7ec-2d0e69ebe8c7 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:09.825982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:09.897802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1026/2038] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:09 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:09.899139 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94d491-d9ea-441d-8f6c-b170dbfc6152 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:09.909922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d94d491-d9ea-441d-8f6c-b170dbfc6152 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:09.972426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b595eb-e7f0-4ed9-8e0c-bc5dc8a26de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:09.973005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b595eb-e7f0-4ed9-8e0c-bc5dc8a26de8 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:09.973281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b595eb-e7f0-4ed9-8e0c-bc5dc8a26de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:09.974108 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b595eb-e7f0-4ed9-8e0c-bc5dc8a26de8 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:09.975014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b595eb-e7f0-4ed9-8e0c-bc5dc8a26de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:09.976925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b595eb-e7f0-4ed9-8e0c-bc5dc8a26de8 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:10.105305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1013/2039] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:09 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:10.109507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e4661-769c-48ae-b7ec-2d0e69ebe8c7 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:10.126342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151e4661-769c-48ae-b7ec-2d0e69ebe8c7 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:10.132324 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880a2fe-0300-48a3-a85d-99e0a73d770d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:10.133047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880a2fe-0300-48a3-a85d-99e0a73d770d admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:10.133550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880a2fe-0300-48a3-a85d-99e0a73d770d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:10.133961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880a2fe-0300-48a3-a85d-99e0a73d770d admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:10.134702 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880a2fe-0300-48a3-a85d-99e0a73d770d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:10.136419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880a2fe-0300-48a3-a85d-99e0a73d770d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:10.147183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1027/2040] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:10.151117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b595eb-e7f0-4ed9-8e0c-bc5dc8a26de8 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:10.163911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b595eb-e7f0-4ed9-8e0c-bc5dc8a26de8 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:10.168365 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186470a5-f229-4628-b0ba-6fdcc48c7357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:10.169958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186470a5-f229-4628-b0ba-6fdcc48c7357 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:10.169958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186470a5-f229-4628-b0ba-6fdcc48c7357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:10.169958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186470a5-f229-4628-b0ba-6fdcc48c7357 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:10.170218 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186470a5-f229-4628-b0ba-6fdcc48c7357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:10.171783 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186470a5-f229-4628-b0ba-6fdcc48c7357 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:10.186515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1014/2041] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:10.187964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880a2fe-0300-48a3-a85d-99e0a73d770d admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:10.198784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880a2fe-0300-48a3-a85d-99e0a73d770d admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:10.203537 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80244635-c5d5-4024-b98b-8c78c97627d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:10.204228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80244635-c5d5-4024-b98b-8c78c97627d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:10.204502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80244635-c5d5-4024-b98b-8c78c97627d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:10.204772 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80244635-c5d5-4024-b98b-8c78c97627d0 admin admin] PATH_INFO: `/v3/projects/2d89086b8d50424485a5a8c28ab285a8/users/4b5b28da1547489f95960e2ebaddd7fb/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:10.212118 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186470a5-f229-4628-b0ba-6fdcc48c7357 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:10.215153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-186470a5-f229-4628-b0ba-6fdcc48c7357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:10.231251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80244635-c5d5-4024-b98b-8c78c97627d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d89086b8d50424485a5a8c28ab285a8, user_id=4b5b28da1547489f95960e2ebaddd7fb, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:10.233463 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80244635-c5d5-4024-b98b-8c78c97627d0 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:10.237351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186470a5-f229-4628-b0ba-6fdcc48c7357 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:10.244630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7f658d-9b63-4831-81b1-635c3afa6553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:10.245420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f658d-9b63-4831-81b1-635c3afa6553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:10.245564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f658d-9b63-4831-81b1-635c3afa6553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:10.246239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7f658d-9b63-4831-81b1-635c3afa6553 admin admin] PATH_INFO: `/v3/users/7ad59abaec3443578372bef1825cd36d` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:10.247462 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f658d-9b63-4831-81b1-635c3afa6553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ad59abaec3443578372bef1825cd36d)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:10.251970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7f658d-9b63-4831-81b1-635c3afa6553 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:10.310809 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1028/2042] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:10 2022] PUT /identity/v3/projects/2d89086b8d50424485a5a8c28ab285a8/users/4b5b28da1547489f95960e2ebaddd7fb/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:10.321426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974e0e5-da0e-4559-82c9-b1aed13a763f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:10.321781 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974e0e5-da0e-4559-82c9-b1aed13a763f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:10.322189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974e0e5-da0e-4559-82c9-b1aed13a763f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:10.359805 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a974e0e5-da0e-4559-82c9-b1aed13a763f None None] MFA Rules not processed for user `4b5b28da1547489f95960e2ebaddd7fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:10.362482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a974e0e5-da0e-4559-82c9-b1aed13a763f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:10.444486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fe7f658d-9b63-4831-81b1-635c3afa6553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ad59abaec3443578372bef1825cd36d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:10.445186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fe7f658d-9b63-4831-81b1-635c3afa6553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ad59abaec3443578372bef1825cd36d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:10.478946 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1029/2043] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:10.498555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974e0e5-da0e-4559-82c9-b1aed13a763f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:10.500145 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a974e0e5-da0e-4559-82c9-b1aed13a763f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:10.527040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fe7f658d-9b63-4831-81b1-635c3afa6553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ad59abaec3443578372bef1825cd36d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:10.540776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fe7f658d-9b63-4831-81b1-635c3afa6553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ad59abaec3443578372bef1825cd36d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:10.541352 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fe7f658d-9b63-4831-81b1-635c3afa6553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ad59abaec3443578372bef1825cd36d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:10.572142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:10.578706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1015/2044] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:10 2022] DELETE /identity/v3/users/7ad59abaec3443578372bef1825cd36d => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:10.584243 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2d9c4-f0b1-47c1-9fda-f31c0b209637 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:10.585443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2d9c4-f0b1-47c1-9fda-f31c0b209637 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:10.585971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2d9c4-f0b1-47c1-9fda-f31c0b209637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:10.594855 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974e0e5-da0e-4559-82c9-b1aed13a763f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:10.613713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-d4e2d9c4-f0b1-47c1-9fda-f31c0b209637 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:10.616034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d4e2d9c4-f0b1-47c1-9fda-f31c0b209637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:10.686969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596cd0f-a765-4202-8373-465fd4e9673f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:10.687604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596cd0f-a765-4202-8373-465fd4e9673f service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:10.687913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596cd0f-a765-4202-8373-465fd4e9673f service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:10.688265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596cd0f-a765-4202-8373-465fd4e9673f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:10.690365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-9596cd0f-a765-4202-8373-465fd4e9673f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:10.711782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:10.719941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1016/2045] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:10.730918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2d9c4-f0b1-47c1-9fda-f31c0b209637 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:10.733327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d4e2d9c4-f0b1-47c1-9fda-f31c0b209637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:10.791608 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596cd0f-a765-4202-8373-465fd4e9673f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:10.793679 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596cd0f-a765-4202-8373-465fd4e9673f service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:10.812930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1030/2046] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:10.851916 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2d9c4-f0b1-47c1-9fda-f31c0b209637 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:10.861464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4625ba2-7207-4556-8469-28c269095860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:10.863351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4625ba2-7207-4556-8469-28c269095860 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:10.863846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4625ba2-7207-4556-8469-28c269095860 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:10.864426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4625ba2-7207-4556-8469-28c269095860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:10.889804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4625ba2-7207-4556-8469-28c269095860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:10.892409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4625ba2-7207-4556-8469-28c269095860 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:10.923388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1017/2047] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:11.407061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596cd0f-a765-4202-8373-465fd4e9673f service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:11.408723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-9596cd0f-a765-4202-8373-465fd4e9673f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:11.440874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596cd0f-a765-4202-8373-465fd4e9673f service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:11.465550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:11.466594 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:11.470056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:11.470464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b admin admin] PATH_INFO: `/v3/projects/c09db6f287454bd08071acbab824eee3` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:11.475997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c09db6f287454bd08071acbab824eee3)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:11.480244 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:11.656964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c09db6f287454bd08071acbab824eee3', 'request_id': 'req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:11.664801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c09db6f287454bd08071acbab824eee3', 'request_id': 'req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:11.677453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c09db6f287454bd08071acbab824eee3', 'request_id': 'req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:11.746235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b admin admin] The token cache is being invalidate because project c09db6f287454bd08071acbab824eee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:11.747089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:11.747982 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9f3c55b4-28f8-4ec4-a9ad-81d87ccaad7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:11.752314 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1031/2048] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:11 2022] DELETE /identity/v3/projects/c09db6f287454bd08071acbab824eee3 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:11.757170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808ca72-e560-452f-bd70-a0eced9afd21 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:11.757662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808ca72-e560-452f-bd70-a0eced9afd21 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:11.757745 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c808ca72-e560-452f-bd70-a0eced9afd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:11.789985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c808ca72-e560-452f-bd70-a0eced9afd21 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:11.796880 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c808ca72-e560-452f-bd70-a0eced9afd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:11.969443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1018/2049] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:11.976670 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a751f-3fa0-46b2-a26b-2bfedca18a95 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:11.976954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a751f-3fa0-46b2-a26b-2bfedca18a95 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:11.977229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803a751f-3fa0-46b2-a26b-2bfedca18a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:11.999766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-803a751f-3fa0-46b2-a26b-2bfedca18a95 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:12.002473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-803a751f-3fa0-46b2-a26b-2bfedca18a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:12.011899 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1032/2050] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:12.015737 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808ca72-e560-452f-bd70-a0eced9afd21 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:12.027428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c808ca72-e560-452f-bd70-a0eced9afd21 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:12.033866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d18d00-9eef-4af4-b184-ebd7c943e461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:12.033866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d18d00-9eef-4af4-b184-ebd7c943e461 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:12.034267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d18d00-9eef-4af4-b184-ebd7c943e461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:12.034430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d18d00-9eef-4af4-b184-ebd7c943e461 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:12.035370 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d18d00-9eef-4af4-b184-ebd7c943e461 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:44:12.035674 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d18d00-9eef-4af4-b184-ebd7c943e461 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:12.038346 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d18d00-9eef-4af4-b184-ebd7c943e461 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:12.055411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1019/2051] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:44:12.056462 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a751f-3fa0-46b2-a26b-2bfedca18a95 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:12.066909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803a751f-3fa0-46b2-a26b-2bfedca18a95 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:12.076522 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9b566-9425-4b0d-87e1-f60b3120e218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:12.077173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9b566-9425-4b0d-87e1-f60b3120e218 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:12.077479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9b566-9425-4b0d-87e1-f60b3120e218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:12.077751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e9b566-9425-4b0d-87e1-f60b3120e218 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:12.079344 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9b566-9425-4b0d-87e1-f60b3120e218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:12.081492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e9b566-9425-4b0d-87e1-f60b3120e218 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:12.171053 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1033/2052] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:12 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:12.175265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d18d00-9eef-4af4-b184-ebd7c943e461 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:12.186901 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d18d00-9eef-4af4-b184-ebd7c943e461 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:12.221800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:12.228545 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9b566-9425-4b0d-87e1-f60b3120e218 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:12.238897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-87e9b566-9425-4b0d-87e1-f60b3120e218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:12.262061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13724a-aeb8-4a3b-b295-e9b643f31e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:12.262721 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13724a-aeb8-4a3b-b295-e9b643f31e52 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:12.262993 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13724a-aeb8-4a3b-b295-e9b643f31e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:12.263256 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13724a-aeb8-4a3b-b295-e9b643f31e52 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:12.267808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13724a-aeb8-4a3b-b295-e9b643f31e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:12.267808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b13724a-aeb8-4a3b-b295-e9b643f31e52 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:12.303026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e9b566-9425-4b0d-87e1-f60b3120e218 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:12.308092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce5058-dd53-4126-9652-0a8b57f0a988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:12.309501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce5058-dd53-4126-9652-0a8b57f0a988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:12.309774 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce5058-dd53-4126-9652-0a8b57f0a988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:12.310076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce5058-dd53-4126-9652-0a8b57f0a988 admin admin] PATH_INFO: `/v3/users/084da259acad41a6b824d1aa7bc9f580` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:12.311340 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce5058-dd53-4126-9652-0a8b57f0a988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=084da259acad41a6b824d1aa7bc9f580)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:12.313249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce5058-dd53-4126-9652-0a8b57f0a988 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:12.324712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:12.351293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1020/2053] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:12 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:44:12.355452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13724a-aeb8-4a3b-b295-e9b643f31e52 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:12.365705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13724a-aeb8-4a3b-b295-e9b643f31e52 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:12.373137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6eae3d-5214-4232-a4f7-2b162806444f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:12.373750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6eae3d-5214-4232-a4f7-2b162806444f admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:12.374056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6eae3d-5214-4232-a4f7-2b162806444f admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:12.374327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6eae3d-5214-4232-a4f7-2b162806444f admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:12.374974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6eae3d-5214-4232-a4f7-2b162806444f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:12.376557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6eae3d-5214-4232-a4f7-2b162806444f admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:12.386712 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1021/2054] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:44:12.392363 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6eae3d-5214-4232-a4f7-2b162806444f admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:12.412800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6eae3d-5214-4232-a4f7-2b162806444f admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:12.418876 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c20715-6218-4527-b80a-e1168792f2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:12.421782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c20715-6218-4527-b80a-e1168792f2ca admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:12.421782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c20715-6218-4527-b80a-e1168792f2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:12.421782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c20715-6218-4527-b80a-e1168792f2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:12.421782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c20715-6218-4527-b80a-e1168792f2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:12.422519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c20715-6218-4527-b80a-e1168792f2ca admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:12.434486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1022/2055] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:12.438785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c20715-6218-4527-b80a-e1168792f2ca admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:12.449342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c20715-6218-4527-b80a-e1168792f2ca admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:12.454832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002a642-94ec-4515-8b0c-5d5f1d16714c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:12.455510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002a642-94ec-4515-8b0c-5d5f1d16714c admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:12.455786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002a642-94ec-4515-8b0c-5d5f1d16714c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:12.456060 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002a642-94ec-4515-8b0c-5d5f1d16714c admin admin] PATH_INFO: `/v3/projects/e81ea6c4c3714416b9d51e152730d997/users/aad40867444049b3b2692d3208b1ba8d/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:12.472943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002a642-94ec-4515-8b0c-5d5f1d16714c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81ea6c4c3714416b9d51e152730d997, user_id=aad40867444049b3b2692d3208b1ba8d, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:12.476444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002a642-94ec-4515-8b0c-5d5f1d16714c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:12.521680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-68ce5058-dd53-4126-9652-0a8b57f0a988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '084da259acad41a6b824d1aa7bc9f580'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:12.530032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-68ce5058-dd53-4126-9652-0a8b57f0a988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '084da259acad41a6b824d1aa7bc9f580'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:12.578870 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1023/2056] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:12 2022] PUT /identity/v3/projects/e81ea6c4c3714416b9d51e152730d997/users/aad40867444049b3b2692d3208b1ba8d/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:12.590570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310f70b-4789-451e-a767-80e945a721bf None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:12.590900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310f70b-4789-451e-a767-80e945a721bf None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:12.591302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310f70b-4789-451e-a767-80e945a721bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:12.595549 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-68ce5058-dd53-4126-9652-0a8b57f0a988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '084da259acad41a6b824d1aa7bc9f580'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:12.596451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-68ce5058-dd53-4126-9652-0a8b57f0a988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '084da259acad41a6b824d1aa7bc9f580'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:12.610240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-68ce5058-dd53-4126-9652-0a8b57f0a988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '084da259acad41a6b824d1aa7bc9f580'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:12.623816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-6310f70b-4789-451e-a767-80e945a721bf None None] MFA Rules not processed for user `aad40867444049b3b2692d3208b1ba8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:12.628624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6310f70b-4789-451e-a767-80e945a721bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:12.632079 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1034/2057] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:12 2022] DELETE /identity/v3/users/084da259acad41a6b824d1aa7bc9f580 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:12.636377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59824c-0380-42d0-bbe8-6c85c53220fd None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:12.636657 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59824c-0380-42d0-bbe8-6c85c53220fd None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:12.636927 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59824c-0380-42d0-bbe8-6c85c53220fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:12.661978 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-4d59824c-0380-42d0-bbe8-6c85c53220fd None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:12.663979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4d59824c-0380-42d0-bbe8-6c85c53220fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:12.699778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1024/2058] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:44:12.705887 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310f70b-4789-451e-a767-80e945a721bf None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:12.707281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6310f70b-4789-451e-a767-80e945a721bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:12.734691 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1035/2059] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:44:12.745054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59824c-0380-42d0-bbe8-6c85c53220fd None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:12.746483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4d59824c-0380-42d0-bbe8-6c85c53220fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:12.783488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310f70b-4789-451e-a767-80e945a721bf None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:12.788272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812e5b8-726f-4628-8923-692af1411b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:12.788919 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812e5b8-726f-4628-8923-692af1411b8d admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:12.789189 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812e5b8-726f-4628-8923-692af1411b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:12.789452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812e5b8-726f-4628-8923-692af1411b8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:12.790128 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812e5b8-726f-4628-8923-692af1411b8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:12.791995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812e5b8-726f-4628-8923-692af1411b8d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:12.803581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:12.825388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59824c-0380-42d0-bbe8-6c85c53220fd None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:12.829716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc04fa-f525-4341-90bb-2bdb0a5f96f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:12.830372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc04fa-f525-4341-90bb-2bdb0a5f96f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:12.832029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc04fa-f525-4341-90bb-2bdb0a5f96f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:12.832393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc04fa-f525-4341-90bb-2bdb0a5f96f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:12.845267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc04fa-f525-4341-90bb-2bdb0a5f96f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:12.847536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc04fa-f525-4341-90bb-2bdb0a5f96f8 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:12.862830 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1036/2060] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:44:12.930273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1025/2061] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:12 2022] POST /identity/v3/projects => generated 375 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:44:12.937285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc04fa-f525-4341-90bb-2bdb0a5f96f8 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:12.956451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc04fa-f525-4341-90bb-2bdb0a5f96f8 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:13.038061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:13.090363 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3acd6-ecf4-41cb-a458-f79c61c0e9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:13.095030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3acd6-ecf4-41cb-a458-f79c61c0e9a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:13.096195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3acd6-ecf4-41cb-a458-f79c61c0e9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:13.096967 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3acd6-ecf4-41cb-a458-f79c61c0e9a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:13.098524 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3acd6-ecf4-41cb-a458-f79c61c0e9a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:13.103866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3acd6-ecf4-41cb-a458-f79c61c0e9a4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:13.216536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1037/2062] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:12 2022] POST /identity/v3/users => generated 294 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:13.221212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812e5b8-726f-4628-8923-692af1411b8d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:13.242101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812e5b8-726f-4628-8923-692af1411b8d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:13.258034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a520a9-457b-4d83-9618-c2fd28a6e662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:13.258922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a520a9-457b-4d83-9618-c2fd28a6e662 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:13.263868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a520a9-457b-4d83-9618-c2fd28a6e662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:13.263868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a520a9-457b-4d83-9618-c2fd28a6e662 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:13.263868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a520a9-457b-4d83-9618-c2fd28a6e662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:13.263868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a520a9-457b-4d83-9618-c2fd28a6e662 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:13.280125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1026/2063] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:13.285550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3acd6-ecf4-41cb-a458-f79c61c0e9a4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:13.297845 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3acd6-ecf4-41cb-a458-f79c61c0e9a4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:13.306825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0beafb3-f210-4462-884c-bf93ce0c7e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:13.307521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0beafb3-f210-4462-884c-bf93ce0c7e5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:13.307802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0beafb3-f210-4462-884c-bf93ce0c7e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:13.308071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0beafb3-f210-4462-884c-bf93ce0c7e5a admin admin] PATH_INFO: `/v3/projects/c709b3977940497bbd8f04a337cdbb96/users/30c4eb49e6fb4807a65dee11e5b6f65e/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:13.331798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0beafb3-f210-4462-884c-bf93ce0c7e5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c709b3977940497bbd8f04a337cdbb96, user_id=30c4eb49e6fb4807a65dee11e5b6f65e, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:13.334973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0beafb3-f210-4462-884c-bf93ce0c7e5a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:13.445009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1038/2064] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:13 2022] PUT /identity/v3/projects/c709b3977940497bbd8f04a337cdbb96/users/30c4eb49e6fb4807a65dee11e5b6f65e/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:13.449578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a520a9-457b-4d83-9618-c2fd28a6e662 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:13.495667 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a520a9-457b-4d83-9618-c2fd28a6e662 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:13.664835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6051660-7b4f-40cb-9146-e0a1768246a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:13.665577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6051660-7b4f-40cb-9146-e0a1768246a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:13.666012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6051660-7b4f-40cb-9146-e0a1768246a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:13.668267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6051660-7b4f-40cb-9146-e0a1768246a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:13.669002 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6051660-7b4f-40cb-9146-e0a1768246a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:13.673451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6051660-7b4f-40cb-9146-e0a1768246a1 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:13.693905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1027/2065] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:13.697287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0beafb3-f210-4462-884c-bf93ce0c7e5a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:13.715960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0beafb3-f210-4462-884c-bf93ce0c7e5a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:13.725577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4aec5-38cd-4136-a911-a7f465f27ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:13.726517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4aec5-38cd-4136-a911-a7f465f27ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:13.726851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4aec5-38cd-4136-a911-a7f465f27ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:13.727196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4aec5-38cd-4136-a911-a7f465f27ed4 admin admin] PATH_INFO: `/v3/projects/c709b3977940497bbd8f04a337cdbb96/users/30c4eb49e6fb4807a65dee11e5b6f65e/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:13.729792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6051660-7b4f-40cb-9146-e0a1768246a1 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:13.729977 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4aec5-38cd-4136-a911-a7f465f27ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c709b3977940497bbd8f04a337cdbb96, user_id=30c4eb49e6fb4807a65dee11e5b6f65e, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:13.731739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c6051660-7b4f-40cb-9146-e0a1768246a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:13.734485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4aec5-38cd-4136-a911-a7f465f27ed4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:13.763938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1039/2066] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:13 2022] PUT /identity/v3/projects/c709b3977940497bbd8f04a337cdbb96/users/30c4eb49e6fb4807a65dee11e5b6f65e/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:13.767833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff791c-3b9e-4812-aef0-0b06e2ca2ab6 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:13.768361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff791c-3b9e-4812-aef0-0b06e2ca2ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:13.768863 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfff791c-3b9e-4812-aef0-0b06e2ca2ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:13.774475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6051660-7b4f-40cb-9146-e0a1768246a1 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:13.838654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-cfff791c-3b9e-4812-aef0-0b06e2ca2ab6 None None] MFA Rules not processed for user `30c4eb49e6fb4807a65dee11e5b6f65e`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:13.839948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-cfff791c-3b9e-4812-aef0-0b06e2ca2ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:13.928810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efb232-bc5f-4437-bb67-3bf48f3f898a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:13.929297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efb232-bc5f-4437-bb67-3bf48f3f898a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:13.929445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efb232-bc5f-4437-bb67-3bf48f3f898a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:13.930141 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efb232-bc5f-4437-bb67-3bf48f3f898a admin admin] PATH_INFO: `/v3/projects/6c60f20274994638a50f21b037541c12` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:13.932612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30efb232-bc5f-4437-bb67-3bf48f3f898a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c60f20274994638a50f21b037541c12)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:13.939902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30efb232-bc5f-4437-bb67-3bf48f3f898a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:14.017623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1040/2067] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:14.029862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff791c-3b9e-4812-aef0-0b06e2ca2ab6 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:14.031322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-cfff791c-3b9e-4812-aef0-0b06e2ca2ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:14.109521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfff791c-3b9e-4812-aef0-0b06e2ca2ab6 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:14.110167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-30efb232-bc5f-4437-bb67-3bf48f3f898a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c60f20274994638a50f21b037541c12', 'request_id': 'req-30efb232-bc5f-4437-bb67-3bf48f3f898a'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.116581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-30efb232-bc5f-4437-bb67-3bf48f3f898a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c60f20274994638a50f21b037541c12', 'request_id': 'req-30efb232-bc5f-4437-bb67-3bf48f3f898a'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.132719 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-30efb232-bc5f-4437-bb67-3bf48f3f898a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c60f20274994638a50f21b037541c12', 'request_id': 'req-30efb232-bc5f-4437-bb67-3bf48f3f898a'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.134760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:14.180417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002da023-0451-4f96-9f00-87ba36f6b9b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:14.181149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002da023-0451-4f96-9f00-87ba36f6b9b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:14.181390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002da023-0451-4f96-9f00-87ba36f6b9b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:14.181701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002da023-0451-4f96-9f00-87ba36f6b9b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:14.201593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-30efb232-bc5f-4437-bb67-3bf48f3f898a admin admin] The token cache is being invalidate because project 6c60f20274994638a50f21b037541c12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:14.201805 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-30efb232-bc5f-4437-bb67-3bf48f3f898a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.202471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-30efb232-bc5f-4437-bb67-3bf48f3f898a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.204227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1028/2068] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:13 2022] DELETE /identity/v3/projects/6c60f20274994638a50f21b037541c12 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:14.211330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c16c18-e028-4fcb-9fb5-32ecb219f711 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:14.212177 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c16c18-e028-4fcb-9fb5-32ecb219f711 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:14.212458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c16c18-e028-4fcb-9fb5-32ecb219f711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:14.222482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:14.235402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c8c16c18-e028-4fcb-9fb5-32ecb219f711 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:14.237290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c8c16c18-e028-4fcb-9fb5-32ecb219f711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:14.269200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002da023-0451-4f96-9f00-87ba36f6b9b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:14.270955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002da023-0451-4f96-9f00-87ba36f6b9b3 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:14.284206 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1041/2069] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:44:14.285190 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002da023-0451-4f96-9f00-87ba36f6b9b3 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:14.286867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-002da023-0451-4f96-9f00-87ba36f6b9b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:14.322824 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1029/2070] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:14.331860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82df205-58df-4d25-9b02-a91518815cb9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:14.332163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82df205-58df-4d25-9b02-a91518815cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:14.332450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82df205-58df-4d25-9b02-a91518815cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:14.363239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-b82df205-58df-4d25-9b02-a91518815cb9 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:14.364968 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b82df205-58df-4d25-9b02-a91518815cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:14.374429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1030/2071] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:44:14.381394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82df205-58df-4d25-9b02-a91518815cb9 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:14.395721 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82df205-58df-4d25-9b02-a91518815cb9 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:14.401953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770da94-1d85-4b47-870b-ca8c91886d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:14.403959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770da94-1d85-4b47-870b-ca8c91886d34 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:14.404185 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770da94-1d85-4b47-870b-ca8c91886d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:14.404452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770da94-1d85-4b47-870b-ca8c91886d34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:14.406949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770da94-1d85-4b47-870b-ca8c91886d34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:44:14.406949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770da94-1d85-4b47-870b-ca8c91886d34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:14.408326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770da94-1d85-4b47-870b-ca8c91886d34 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:14.415305 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002da023-0451-4f96-9f00-87ba36f6b9b3 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:14.421407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1031/2072] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:44:14.422731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:14.429242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:14.429242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:14.429242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] PATH_INFO: `/v3/users/62e3850e3d554f3f94d9afeedfd26d7d` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:14.429242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e3850e3d554f3f94d9afeedfd26d7d)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:14.429242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:14.443564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770da94-1d85-4b47-870b-ca8c91886d34 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:14.454036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770da94-1d85-4b47-870b-ca8c91886d34 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:14.459223 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7abfe-8892-4edb-991d-319fc156d387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:14.459836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7abfe-8892-4edb-991d-319fc156d387 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:14.460106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7abfe-8892-4edb-991d-319fc156d387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:14.460366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7abfe-8892-4edb-991d-319fc156d387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:14.460997 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7abfe-8892-4edb-991d-319fc156d387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:14.462987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7abfe-8892-4edb-991d-319fc156d387 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:14.597162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1032/2073] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:14 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:14.598154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7abfe-8892-4edb-991d-319fc156d387 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:14.600272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-34d7abfe-8892-4edb-991d-319fc156d387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:14.629397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e3850e3d554f3f94d9afeedfd26d7d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.630225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e3850e3d554f3f94d9afeedfd26d7d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.670710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7abfe-8892-4edb-991d-319fc156d387 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:14.675473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:14.676092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:14.676404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:14.677294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] PATH_INFO: `/v3/users/8e28d24465df4f319bc07406586789fa` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:14.678717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e28d24465df4f319bc07406586789fa)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:14.680654 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:14.732482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e3850e3d554f3f94d9afeedfd26d7d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.733138 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e3850e3d554f3f94d9afeedfd26d7d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.742255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e3850e3d554f3f94d9afeedfd26d7d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.759239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1042/2074] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:14 2022] DELETE /identity/v3/users/62e3850e3d554f3f94d9afeedfd26d7d => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:14.760225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:14.761617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:14.845650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd676c-41e2-4700-b844-b4eadb24865b admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:14.849238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:14.854471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf78bc-c3ef-4332-83be-95c482b0345e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:14.856896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf78bc-c3ef-4332-83be-95c482b0345e admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:14.856896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf78bc-c3ef-4332-83be-95c482b0345e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:14.856896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf78bc-c3ef-4332-83be-95c482b0345e admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:14.856896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf78bc-c3ef-4332-83be-95c482b0345e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:14.858320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf78bc-c3ef-4332-83be-95c482b0345e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:14.920917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e28d24465df4f319bc07406586789fa'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.921591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e28d24465df4f319bc07406586789fa'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.945256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1043/2075] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:14 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:14.946554 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc8877-49ac-45b2-9fca-c28994e933eb None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:14.946832 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc8877-49ac-45b2-9fca-c28994e933eb None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:14.947108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfc8877-49ac-45b2-9fca-c28994e933eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:14.978697 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e28d24465df4f319bc07406586789fa'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.979813 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-fcfc8877-49ac-45b2-9fca-c28994e933eb None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:14.981185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-fcfc8877-49ac-45b2-9fca-c28994e933eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:14.989016 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e28d24465df4f319bc07406586789fa'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.990256 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e28d24465df4f319bc07406586789fa'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:14.992685 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1044/2076] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:14.997731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc8877-49ac-45b2-9fca-c28994e933eb None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:15.016904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfc8877-49ac-45b2-9fca-c28994e933eb None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:15.030916 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1033/2077] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:14 2022] DELETE /identity/v3/users/8e28d24465df4f319bc07406586789fa => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:15.031772 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:15.032877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16def299-340d-4a19-8f52-c3e8dba992b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:15.033499 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16def299-340d-4a19-8f52-c3e8dba992b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:15.033779 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16def299-340d-4a19-8f52-c3e8dba992b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:15.034362 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16def299-340d-4a19-8f52-c3e8dba992b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:15.035022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16def299-340d-4a19-8f52-c3e8dba992b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:15.037357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16def299-340d-4a19-8f52-c3e8dba992b5 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:15.039456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:15.061242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1045/2078] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:14 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:15.065910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16def299-340d-4a19-8f52-c3e8dba992b5 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:15.075701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-16def299-340d-4a19-8f52-c3e8dba992b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:15.098436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16def299-340d-4a19-8f52-c3e8dba992b5 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:15.124693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:15.141708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a30c0d-c3e5-4b2d-b095-bc0c4d61de82 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:15.146334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473f844-bd99-4a6b-a9de-34daba9a2b10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:15.147067 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473f844-bd99-4a6b-a9de-34daba9a2b10 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:15.147408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473f844-bd99-4a6b-a9de-34daba9a2b10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:15.147748 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473f844-bd99-4a6b-a9de-34daba9a2b10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:15.149793 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0473f844-bd99-4a6b-a9de-34daba9a2b10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:15.251429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86d0a2-0079-4231-9429-41b00aa3bd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:15.252278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86d0a2-0079-4231-9429-41b00aa3bd48 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:15.252823 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86d0a2-0079-4231-9429-41b00aa3bd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:15.253314 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86d0a2-0079-4231-9429-41b00aa3bd48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:15.254189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86d0a2-0079-4231-9429-41b00aa3bd48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:15.257399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86d0a2-0079-4231-9429-41b00aa3bd48 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:15.258650 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473f844-bd99-4a6b-a9de-34daba9a2b10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:15.260183 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473f844-bd99-4a6b-a9de-34daba9a2b10 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:15.274748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1046/2079] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:15.279440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86d0a2-0079-4231-9429-41b00aa3bd48 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:15.284061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1034/2080] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:15.291255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86d0a2-0079-4231-9429-41b00aa3bd48 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:15.306910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ecc0d-5608-4c0a-95ab-ddebd816b0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:15.306910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ecc0d-5608-4c0a-95ab-ddebd816b0f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:15.306910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ecc0d-5608-4c0a-95ab-ddebd816b0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:15.306910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ecc0d-5608-4c0a-95ab-ddebd816b0f6 admin admin] PATH_INFO: `/v3/projects/19b1f84056d6452ca3abac0d8530f2b8/users/d10ee696eff649b5ae4631a048021d0c/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:15.329743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ecc0d-5608-4c0a-95ab-ddebd816b0f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b1f84056d6452ca3abac0d8530f2b8, user_id=d10ee696eff649b5ae4631a048021d0c, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:15.333028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ecc0d-5608-4c0a-95ab-ddebd816b0f6 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:15.458488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1047/2081] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:15 2022] PUT /identity/v3/projects/19b1f84056d6452ca3abac0d8530f2b8/users/d10ee696eff649b5ae4631a048021d0c/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:15.469615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21361e28-640f-4467-a9c4-59c9aeb1ca7f None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:15.469926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21361e28-640f-4467-a9c4-59c9aeb1ca7f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:15.470199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21361e28-640f-4467-a9c4-59c9aeb1ca7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:15.498098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-21361e28-640f-4467-a9c4-59c9aeb1ca7f None None] MFA Rules not processed for user `d10ee696eff649b5ae4631a048021d0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:15.499422 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-21361e28-640f-4467-a9c4-59c9aeb1ca7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:15.510356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ecc0d-5608-4c0a-95ab-ddebd816b0f6 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:15.512407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d84ecc0d-5608-4c0a-95ab-ddebd816b0f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:15.578693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:15.599169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ecc0d-5608-4c0a-95ab-ddebd816b0f6 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:15.604484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe64d1-0408-4e43-93d7-05224f2b3d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:15.605118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe64d1-0408-4e43-93d7-05224f2b3d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:15.605388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe64d1-0408-4e43-93d7-05224f2b3d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:15.605786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe64d1-0408-4e43-93d7-05224f2b3d84 admin admin] PATH_INFO: `/v3/projects/3d0323b2af5040ee976e6a0a223ebe2c` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:15.606981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe64d1-0408-4e43-93d7-05224f2b3d84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d0323b2af5040ee976e6a0a223ebe2c)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:15.608876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe64d1-0408-4e43-93d7-05224f2b3d84 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:15.623377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1035/2082] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:15.739369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-77fe64d1-0408-4e43-93d7-05224f2b3d84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d0323b2af5040ee976e6a0a223ebe2c', 'request_id': 'req-77fe64d1-0408-4e43-93d7-05224f2b3d84'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:15.745199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-77fe64d1-0408-4e43-93d7-05224f2b3d84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d0323b2af5040ee976e6a0a223ebe2c', 'request_id': 'req-77fe64d1-0408-4e43-93d7-05224f2b3d84'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:15.751393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-77fe64d1-0408-4e43-93d7-05224f2b3d84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d0323b2af5040ee976e6a0a223ebe2c', 'request_id': 'req-77fe64d1-0408-4e43-93d7-05224f2b3d84'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:15.786095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21361e28-640f-4467-a9c4-59c9aeb1ca7f None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:15.788338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-21361e28-640f-4467-a9c4-59c9aeb1ca7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:15.805335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21361e28-640f-4467-a9c4-59c9aeb1ca7f None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:15.824777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-77fe64d1-0408-4e43-93d7-05224f2b3d84 admin admin] The token cache is being invalidate because project 3d0323b2af5040ee976e6a0a223ebe2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:15.825916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-77fe64d1-0408-4e43-93d7-05224f2b3d84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:15.826040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:15.826233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-77fe64d1-0408-4e43-93d7-05224f2b3d84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:15.827811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1048/2083] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:15 2022] DELETE /identity/v3/projects/3d0323b2af5040ee976e6a0a223ebe2c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:15.832562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b01171-b226-4106-b2ac-da93fdfe8311 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:15.832842 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b01171-b226-4106-b2ac-da93fdfe8311 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:15.833106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b01171-b226-4106-b2ac-da93fdfe8311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:15.855945 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-82b01171-b226-4106-b2ac-da93fdfe8311 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:15.857764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-82b01171-b226-4106-b2ac-da93fdfe8311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:15.887743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5e572-c53b-4cca-b94b-a48637d4062d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:15.888366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5e572-c53b-4cca-b94b-a48637d4062d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:15.888637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5e572-c53b-4cca-b94b-a48637d4062d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:15.888937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5e572-c53b-4cca-b94b-a48637d4062d admin admin] PATH_INFO: `/v3/projects/410c3bb930d74d02807b0db4ab32c50e` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:15.890104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5e572-c53b-4cca-b94b-a48637d4062d admin admin] RBAC: Authorizing `identity:delete_project(project_id=410c3bb930d74d02807b0db4ab32c50e)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:15.892023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5e572-c53b-4cca-b94b-a48637d4062d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:15.955766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1049/2084] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:15.955766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b01171-b226-4106-b2ac-da93fdfe8311 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:15.960467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-82b01171-b226-4106-b2ac-da93fdfe8311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:16.032374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-66e5e572-c53b-4cca-b94b-a48637d4062d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '410c3bb930d74d02807b0db4ab32c50e', 'request_id': 'req-66e5e572-c53b-4cca-b94b-a48637d4062d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.040277 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-66e5e572-c53b-4cca-b94b-a48637d4062d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '410c3bb930d74d02807b0db4ab32c50e', 'request_id': 'req-66e5e572-c53b-4cca-b94b-a48637d4062d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.042725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b01171-b226-4106-b2ac-da93fdfe8311 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:16.049151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-66e5e572-c53b-4cca-b94b-a48637d4062d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '410c3bb930d74d02807b0db4ab32c50e', 'request_id': 'req-66e5e572-c53b-4cca-b94b-a48637d4062d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.075336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:16.093685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-66e5e572-c53b-4cca-b94b-a48637d4062d admin admin] The token cache is being invalidate because project 410c3bb930d74d02807b0db4ab32c50e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:16.094066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-66e5e572-c53b-4cca-b94b-a48637d4062d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.094804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-66e5e572-c53b-4cca-b94b-a48637d4062d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.101020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1036/2085] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:15 2022] DELETE /identity/v3/projects/410c3bb930d74d02807b0db4ab32c50e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:16.101020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76898e-918b-4490-a378-55250810ca7e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:16.101020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76898e-918b-4490-a378-55250810ca7e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:16.101020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76898e-918b-4490-a378-55250810ca7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:16.105712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f899ec9-c0ec-4ef4-b316-aeb950795892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:16.106556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f899ec9-c0ec-4ef4-b316-aeb950795892 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:16.106752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f899ec9-c0ec-4ef4-b316-aeb950795892 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:16.107189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f899ec9-c0ec-4ef4-b316-aeb950795892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:16.109341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5f899ec9-c0ec-4ef4-b316-aeb950795892 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:16.124563 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-aa76898e-918b-4490-a378-55250810ca7e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:16.126434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-aa76898e-918b-4490-a378-55250810ca7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:16.185682 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f899ec9-c0ec-4ef4-b316-aeb950795892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:16.187089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f899ec9-c0ec-4ef4-b316-aeb950795892 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:16.202256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1050/2086] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:16.203380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1037/2087] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:16.203632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f899ec9-c0ec-4ef4-b316-aeb950795892 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:16.205801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5f899ec9-c0ec-4ef4-b316-aeb950795892 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:16.208269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76898e-918b-4490-a378-55250810ca7e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:16.223956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76898e-918b-4490-a378-55250810ca7e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:16.228926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb087c8-77fd-4392-98b6-877b047b0b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:16.229653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb087c8-77fd-4392-98b6-877b047b0b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:16.230019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb087c8-77fd-4392-98b6-877b047b0b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:16.230351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb087c8-77fd-4392-98b6-877b047b0b23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:16.230618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f899ec9-c0ec-4ef4-b316-aeb950795892 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:16.231132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb087c8-77fd-4392-98b6-877b047b0b23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:44:16.231483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb087c8-77fd-4392-98b6-877b047b0b23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:16.233465 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb087c8-77fd-4392-98b6-877b047b0b23 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:16.240594 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:16.241601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:16.246457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:16.246457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] PATH_INFO: `/v3/users/93bca6ca031c4cd3bfb3aa792ed4ba02` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:16.246457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=93bca6ca031c4cd3bfb3aa792ed4ba02)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:16.247684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:16.247970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1038/2088] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:44:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:44:16.252232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb087c8-77fd-4392-98b6-877b047b0b23 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:16.264097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb087c8-77fd-4392-98b6-877b047b0b23 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:16.269044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003095d9-a7ca-43b6-8b97-11b0f023050a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:16.270549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003095d9-a7ca-43b6-8b97-11b0f023050a admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:16.270812 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003095d9-a7ca-43b6-8b97-11b0f023050a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:16.271079 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003095d9-a7ca-43b6-8b97-11b0f023050a admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:16.271717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003095d9-a7ca-43b6-8b97-11b0f023050a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:16.273652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003095d9-a7ca-43b6-8b97-11b0f023050a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:16.406848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1039/2089] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:16 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:44:16.406848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003095d9-a7ca-43b6-8b97-11b0f023050a admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:16.406848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-003095d9-a7ca-43b6-8b97-11b0f023050a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:16.430060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93bca6ca031c4cd3bfb3aa792ed4ba02'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.430857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93bca6ca031c4cd3bfb3aa792ed4ba02'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.433421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:16.478099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003095d9-a7ca-43b6-8b97-11b0f023050a admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:16.481124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93bca6ca031c4cd3bfb3aa792ed4ba02'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.482216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93bca6ca031c4cd3bfb3aa792ed4ba02'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.482988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4412714-2216-4b81-93f5-5943b002b24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:16.483612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4412714-2216-4b81-93f5-5943b002b24c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:16.483882 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4412714-2216-4b81-93f5-5943b002b24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:16.484147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4412714-2216-4b81-93f5-5943b002b24c admin admin] PATH_INFO: `/v3/users/d10ee696eff649b5ae4631a048021d0c` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:16.485258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4412714-2216-4b81-93f5-5943b002b24c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d10ee696eff649b5ae4631a048021d0c)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:16.487289 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4412714-2216-4b81-93f5-5943b002b24c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:16.489991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93bca6ca031c4cd3bfb3aa792ed4ba02'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.507891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1051/2090] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:16 2022] DELETE /identity/v3/users/93bca6ca031c4cd3bfb3aa792ed4ba02 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:16.508842 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:16.510206 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:16.584456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89202df2-f679-4ed4-bdb9-59e2c44227bc admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:16.589186 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5264132-638a-44be-86e8-6eee3feb66f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:16.590015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5264132-638a-44be-86e8-6eee3feb66f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:16.590132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5264132-638a-44be-86e8-6eee3feb66f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:16.590416 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5264132-638a-44be-86e8-6eee3feb66f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:16.591047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5264132-638a-44be-86e8-6eee3feb66f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:16.592956 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5264132-638a-44be-86e8-6eee3feb66f4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:16.615902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d4412714-2216-4b81-93f5-5943b002b24c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd10ee696eff649b5ae4631a048021d0c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.616825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d4412714-2216-4b81-93f5-5943b002b24c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd10ee696eff649b5ae4631a048021d0c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.635403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:16.662233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1052/2091] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:16 2022] POST /identity/v3/users => generated 303 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:16.663320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5264132-638a-44be-86e8-6eee3feb66f4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:16.669449 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d4412714-2216-4b81-93f5-5943b002b24c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd10ee696eff649b5ae4631a048021d0c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.674320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5264132-638a-44be-86e8-6eee3feb66f4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:16.678964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d4412714-2216-4b81-93f5-5943b002b24c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd10ee696eff649b5ae4631a048021d0c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.679707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d4412714-2216-4b81-93f5-5943b002b24c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd10ee696eff649b5ae4631a048021d0c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:16.679934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31d19f-f057-4f87-854a-ac49c664877e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:16.680571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31d19f-f057-4f87-854a-ac49c664877e admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:16.682614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31d19f-f057-4f87-854a-ac49c664877e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:16.682614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31d19f-f057-4f87-854a-ac49c664877e admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:16.682614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31d19f-f057-4f87-854a-ac49c664877e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:16.683435 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31d19f-f057-4f87-854a-ac49c664877e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:16.697539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1053/2092] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:16.699494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31d19f-f057-4f87-854a-ac49c664877e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:16.701553 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-bf31d19f-f057-4f87-854a-ac49c664877e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:16.709944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1040/2093] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:16 2022] DELETE /identity/v3/users/d10ee696eff649b5ae4631a048021d0c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:16.711602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904d1b0-01d7-4376-bc74-44bb0077e56d None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:16.711899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904d1b0-01d7-4376-bc74-44bb0077e56d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:16.712242 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904d1b0-01d7-4376-bc74-44bb0077e56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:16.715366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31d19f-f057-4f87-854a-ac49c664877e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:16.734300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-4904d1b0-01d7-4376-bc74-44bb0077e56d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:16.735583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4904d1b0-01d7-4376-bc74-44bb0077e56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:16.786856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18a1f0-f5ff-46f5-9b69-4fac03700210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:16.787468 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18a1f0-f5ff-46f5-9b69-4fac03700210 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:16.787743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18a1f0-f5ff-46f5-9b69-4fac03700210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:16.788010 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18a1f0-f5ff-46f5-9b69-4fac03700210 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:16.788653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18a1f0-f5ff-46f5-9b69-4fac03700210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:16.790308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18a1f0-f5ff-46f5-9b69-4fac03700210 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:16.805970 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1054/2094] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:16.806955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1041/2095] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:16.807277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18a1f0-f5ff-46f5-9b69-4fac03700210 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:16.816640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904d1b0-01d7-4376-bc74-44bb0077e56d None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:16.818316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4904d1b0-01d7-4376-bc74-44bb0077e56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:16.819357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18a1f0-f5ff-46f5-9b69-4fac03700210 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:16.824603 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f40426-3910-4e2e-b359-f57629a12fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:16.825266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f40426-3910-4e2e-b359-f57629a12fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:16.825544 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f40426-3910-4e2e-b359-f57629a12fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:16.825836 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f40426-3910-4e2e-b359-f57629a12fa0 admin admin] PATH_INFO: `/v3/projects/67626fcccd8a44e4af89f3193ae20bf7/users/5b3dc71b4413469abf80a33687bc927f/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:16.844148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f40426-3910-4e2e-b359-f57629a12fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67626fcccd8a44e4af89f3193ae20bf7, user_id=5b3dc71b4413469abf80a33687bc927f, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:16.846847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f40426-3910-4e2e-b359-f57629a12fa0 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:16.869739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:16.885512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904d1b0-01d7-4376-bc74-44bb0077e56d None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:16.889154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe600f-618a-4f28-8470-ec8cdfe425c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:16.889762 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe600f-618a-4f28-8470-ec8cdfe425c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:16.890068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe600f-618a-4f28-8470-ec8cdfe425c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:16.890336 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe600f-618a-4f28-8470-ec8cdfe425c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:16.899949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe600f-618a-4f28-8470-ec8cdfe425c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:16.901338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe600f-618a-4f28-8470-ec8cdfe425c4 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:16.911266 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1042/2096] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:44:16.948456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1055/2097] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:16 2022] PUT /identity/v3/projects/67626fcccd8a44e4af89f3193ae20bf7/users/5b3dc71b4413469abf80a33687bc927f/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:16.952019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe600f-618a-4f28-8470-ec8cdfe425c4 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:16.952557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58d020-b1fc-4145-9a0b-e23e961e085e None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:16.952980 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58d020-b1fc-4145-9a0b-e23e961e085e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:16.953608 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d58d020-b1fc-4145-9a0b-e23e961e085e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:16.953763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-babe600f-618a-4f28-8470-ec8cdfe425c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:16.976410 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe600f-618a-4f28-8470-ec8cdfe425c4 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:16.990069 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-8d58d020-b1fc-4145-9a0b-e23e961e085e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:16.993679 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8d58d020-b1fc-4145-9a0b-e23e961e085e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:17.085975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845128f9-5acf-4f65-ab3f-4849192dde2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:17.086610 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845128f9-5acf-4f65-ab3f-4849192dde2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:17.087176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845128f9-5acf-4f65-ab3f-4849192dde2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:17.090167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845128f9-5acf-4f65-ab3f-4849192dde2b admin admin] PATH_INFO: `/v3/projects/1e8c1ae7c0dd4dd383327d4b94a5b9ac` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:17.091925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845128f9-5acf-4f65-ab3f-4849192dde2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8c1ae7c0dd4dd383327d4b94a5b9ac)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:17.102728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845128f9-5acf-4f65-ab3f-4849192dde2b admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:17.111080 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1056/2098] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:17.133846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58d020-b1fc-4145-9a0b-e23e961e085e None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:17.135637 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8d58d020-b1fc-4145-9a0b-e23e961e085e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:17.202321 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:17.215001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d58d020-b1fc-4145-9a0b-e23e961e085e None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:17.219009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f487b1-2f24-481a-9166-a30c62150670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:17.219857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f487b1-2f24-481a-9166-a30c62150670 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:17.220015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f487b1-2f24-481a-9166-a30c62150670 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:17.220415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f487b1-2f24-481a-9166-a30c62150670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:17.237563 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-845128f9-5acf-4f65-ab3f-4849192dde2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8c1ae7c0dd4dd383327d4b94a5b9ac', 'request_id': 'req-845128f9-5acf-4f65-ab3f-4849192dde2b'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.245216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f487b1-2f24-481a-9166-a30c62150670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:17.247406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f487b1-2f24-481a-9166-a30c62150670 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:17.252850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-845128f9-5acf-4f65-ab3f-4849192dde2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8c1ae7c0dd4dd383327d4b94a5b9ac', 'request_id': 'req-845128f9-5acf-4f65-ab3f-4849192dde2b'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.260548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1057/2099] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:17.263265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-845128f9-5acf-4f65-ab3f-4849192dde2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8c1ae7c0dd4dd383327d4b94a5b9ac', 'request_id': 'req-845128f9-5acf-4f65-ab3f-4849192dde2b'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.346986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-845128f9-5acf-4f65-ab3f-4849192dde2b admin admin] The token cache is being invalidate because project 1e8c1ae7c0dd4dd383327d4b94a5b9ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:17.346986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-845128f9-5acf-4f65-ab3f-4849192dde2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.347654 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-845128f9-5acf-4f65-ab3f-4849192dde2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.350332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1043/2100] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:16 2022] DELETE /identity/v3/projects/1e8c1ae7c0dd4dd383327d4b94a5b9ac => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:17.356411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d72ccd-7e17-4830-8749-45088ed4f9cf None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:17.356692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d72ccd-7e17-4830-8749-45088ed4f9cf None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:17.356964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d72ccd-7e17-4830-8749-45088ed4f9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:17.379014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-91d72ccd-7e17-4830-8749-45088ed4f9cf None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:17.383004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-91d72ccd-7e17-4830-8749-45088ed4f9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:17.481984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1058/2101] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:17.492051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb081d0c-7dda-4e88-9c4c-691f5f8c13f9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:17.506893 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb081d0c-7dda-4e88-9c4c-691f5f8c13f9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:17.507084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d72ccd-7e17-4830-8749-45088ed4f9cf None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:17.507254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb081d0c-7dda-4e88-9c4c-691f5f8c13f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:17.509580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-91d72ccd-7e17-4830-8749-45088ed4f9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:17.530824 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-fb081d0c-7dda-4e88-9c4c-691f5f8c13f9 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:17.531068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d72ccd-7e17-4830-8749-45088ed4f9cf None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:17.531158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-fb081d0c-7dda-4e88-9c4c-691f5f8c13f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:17.538113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:17.543107 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1044/2102] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:17.544559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb081d0c-7dda-4e88-9c4c-691f5f8c13f9 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:17.545998 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-fb081d0c-7dda-4e88-9c4c-691f5f8c13f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:17.546882 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:17.547109 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:17.547288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] PATH_INFO: `/v3/projects/19b1f84056d6452ca3abac0d8530f2b8` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:17.549083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b1f84056d6452ca3abac0d8530f2b8)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:17.551141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:17.574930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb081d0c-7dda-4e88-9c4c-691f5f8c13f9 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:17.583568 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:17.584410 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:17.584816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:17.585082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] PATH_INFO: `/v3/users/aad40867444049b3b2692d3208b1ba8d` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:17.587020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] RBAC: Authorizing `identity:delete_user(user_id=aad40867444049b3b2692d3208b1ba8d)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:17.590356 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:17.700601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b1f84056d6452ca3abac0d8530f2b8', 'request_id': 'req-22c541fe-3c89-4080-a083-fde52e4a351f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.706940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b1f84056d6452ca3abac0d8530f2b8', 'request_id': 'req-22c541fe-3c89-4080-a083-fde52e4a351f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.715244 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b1f84056d6452ca3abac0d8530f2b8', 'request_id': 'req-22c541fe-3c89-4080-a083-fde52e4a351f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.766015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aad40867444049b3b2692d3208b1ba8d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.766996 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aad40867444049b3b2692d3208b1ba8d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.778836 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] The token cache is being invalidate because project 19b1f84056d6452ca3abac0d8530f2b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:17.779401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.780321 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.782700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1059/2103] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:17 2022] DELETE /identity/v3/projects/19b1f84056d6452ca3abac0d8530f2b8 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:17.783567 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:17.785097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:17.864072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aad40867444049b3b2692d3208b1ba8d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.864799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c541fe-3c89-4080-a083-fde52e4a351f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:17.872051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:17.872315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205f562-b907-4550-b7b7-924331ec557b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:17.872658 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f562-b907-4550-b7b7-924331ec557b admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:17.872808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f562-b907-4550-b7b7-924331ec557b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:17.873222 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f562-b907-4550-b7b7-924331ec557b admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:17.873923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205f562-b907-4550-b7b7-924331ec557b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:44:17.874045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aad40867444049b3b2692d3208b1ba8d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.874226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205f562-b907-4550-b7b7-924331ec557b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:17.876165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205f562-b907-4550-b7b7-924331ec557b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:17.882302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aad40867444049b3b2692d3208b1ba8d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:17.892449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1060/2104] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:44:17.892449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaefd42-dc9b-41db-a607-f613b7f52586 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:17.892449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaefd42-dc9b-41db-a607-f613b7f52586 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:17.892449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaefd42-dc9b-41db-a607-f613b7f52586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:17.909492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:17.917744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-ecaefd42-dc9b-41db-a607-f613b7f52586 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:17.918970 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ecaefd42-dc9b-41db-a607-f613b7f52586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:17.920354 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1045/2105] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:17 2022] DELETE /identity/v3/users/aad40867444049b3b2692d3208b1ba8d => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:17.921786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:17.923583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:17.996411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1061/2106] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:17.998173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba15fa6-d524-4967-ba5f-ef654b2dceab None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:17.998810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba15fa6-d524-4967-ba5f-ef654b2dceab None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:17.998810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba15fa6-d524-4967-ba5f-ef654b2dceab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.024805 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-cba15fa6-d524-4967-ba5f-ef654b2dceab None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:18.026157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-cba15fa6-d524-4967-ba5f-ef654b2dceab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:18.027410 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec53adf-dd80-489a-b753-53673d8beacb admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:18.032771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48045cb2-7b03-4522-b240-b44e7a84a9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:18.033441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48045cb2-7b03-4522-b240-b44e7a84a9ad admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.033660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48045cb2-7b03-4522-b240-b44e7a84a9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.034209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48045cb2-7b03-4522-b240-b44e7a84a9ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.035174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48045cb2-7b03-4522-b240-b44e7a84a9ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:18.037871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1062/2107] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:18.038021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48045cb2-7b03-4522-b240-b44e7a84a9ad admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:18.038359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882bc18-7736-40c6-95f4-65a2c9f30405 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.038635 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882bc18-7736-40c6-95f4-65a2c9f30405 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.038902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882bc18-7736-40c6-95f4-65a2c9f30405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.061898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-5882bc18-7736-40c6-95f4-65a2c9f30405 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:18.064805 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5882bc18-7736-40c6-95f4-65a2c9f30405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:18.071220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1063/2108] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:18.072015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882bc18-7736-40c6-95f4-65a2c9f30405 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:18.075547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5882bc18-7736-40c6-95f4-65a2c9f30405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:18.131584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1046/2109] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:17 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:18.131584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48045cb2-7b03-4522-b240-b44e7a84a9ad admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:18.145903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882bc18-7736-40c6-95f4-65a2c9f30405 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:18.211932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec21543-ef58-4a00-a957-3955a65a5fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:18.213238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec21543-ef58-4a00-a957-3955a65a5fad service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.214408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec21543-ef58-4a00-a957-3955a65a5fad service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.214980 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec21543-ef58-4a00-a957-3955a65a5fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.231307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48045cb2-7b03-4522-b240-b44e7a84a9ad admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:18.239370 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2115fd-5ecd-4fe7-956a-84bb56fa93e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:18.240491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2115fd-5ecd-4fe7-956a-84bb56fa93e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.242885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2115fd-5ecd-4fe7-956a-84bb56fa93e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.242885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2115fd-5ecd-4fe7-956a-84bb56fa93e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.242885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2115fd-5ecd-4fe7-956a-84bb56fa93e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:44:18.242885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2115fd-5ecd-4fe7-956a-84bb56fa93e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:18.244287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2115fd-5ecd-4fe7-956a-84bb56fa93e5 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:18.275945 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1047/2110] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:44:18.277765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2115fd-5ecd-4fe7-956a-84bb56fa93e5 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:18.293073 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2115fd-5ecd-4fe7-956a-84bb56fa93e5 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:18.298396 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2da4e5-99ec-46ca-9556-06b77d6265ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:18.299007 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2da4e5-99ec-46ca-9556-06b77d6265ff admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.299279 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2da4e5-99ec-46ca-9556-06b77d6265ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.299693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2da4e5-99ec-46ca-9556-06b77d6265ff admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.300322 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2da4e5-99ec-46ca-9556-06b77d6265ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:18.303671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2da4e5-99ec-46ca-9556-06b77d6265ff admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:18.336399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec21543-ef58-4a00-a957-3955a65a5fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:18.338168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec21543-ef58-4a00-a957-3955a65a5fad service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:18.344660 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:18.350848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1064/2111] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:18.351732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec21543-ef58-4a00-a957-3955a65a5fad service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:18.373416 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec21543-ef58-4a00-a957-3955a65a5fad service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:18.378673 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d2b56-054d-414d-bf10-af95b5739242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:18.379956 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d2b56-054d-414d-bf10-af95b5739242 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.380299 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d2b56-054d-414d-bf10-af95b5739242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.380625 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d2b56-054d-414d-bf10-af95b5739242 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.381893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d2b56-054d-414d-bf10-af95b5739242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:18.383420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d2b56-054d-414d-bf10-af95b5739242 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:18.401568 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1048/2112] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:18 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:18.405138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2da4e5-99ec-46ca-9556-06b77d6265ff admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:18.418057 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2da4e5-99ec-46ca-9556-06b77d6265ff admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:18.424767 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf36baa-7415-4a34-ac7a-5d53c3c2333b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:18.425487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf36baa-7415-4a34-ac7a-5d53c3c2333b admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.429602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf36baa-7415-4a34-ac7a-5d53c3c2333b admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.429602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf36baa-7415-4a34-ac7a-5d53c3c2333b admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.430335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf36baa-7415-4a34-ac7a-5d53c3c2333b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:18.432725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf36baa-7415-4a34-ac7a-5d53c3c2333b admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:18.449898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1049/2113] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:18.451559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf36baa-7415-4a34-ac7a-5d53c3c2333b admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:18.462392 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf36baa-7415-4a34-ac7a-5d53c3c2333b admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:18.467833 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59a0ce-61d7-4537-95e7-bbf137df1383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:18.468498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59a0ce-61d7-4537-95e7-bbf137df1383 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.468898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59a0ce-61d7-4537-95e7-bbf137df1383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.469151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59a0ce-61d7-4537-95e7-bbf137df1383 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.469798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59a0ce-61d7-4537-95e7-bbf137df1383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:18.471470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59a0ce-61d7-4537-95e7-bbf137df1383 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:18.484740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1050/2114] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:44:18.488722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59a0ce-61d7-4537-95e7-bbf137df1383 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:18.501751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59a0ce-61d7-4537-95e7-bbf137df1383 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:18.509295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2128c-a015-4923-b0f2-6866f578c481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:18.510360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2128c-a015-4923-b0f2-6866f578c481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.510633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2128c-a015-4923-b0f2-6866f578c481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.510904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2128c-a015-4923-b0f2-6866f578c481 admin admin] PATH_INFO: `/v3/projects/8325bc6ee4d3473e969b8c06de699f50/users/be4b02d5ae954712a5924bd9c2ad3555/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.531150 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1065/2115] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:18 2022] POST /identity/v3/projects => generated 371 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:18.538947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d2b56-054d-414d-bf10-af95b5739242 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:18.542897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:18.559146 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2128c-a015-4923-b0f2-6866f578c481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8325bc6ee4d3473e969b8c06de699f50, user_id=be4b02d5ae954712a5924bd9c2ad3555, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:18.566467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2128c-a015-4923-b0f2-6866f578c481 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:18.619514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d2b56-054d-414d-bf10-af95b5739242 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:18.624498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74cd53-6fad-4a5c-b226-25eb178e8a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:18.625274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74cd53-6fad-4a5c-b226-25eb178e8a53 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.625454 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74cd53-6fad-4a5c-b226-25eb178e8a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.625861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74cd53-6fad-4a5c-b226-25eb178e8a53 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.626431 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74cd53-6fad-4a5c-b226-25eb178e8a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:18.628499 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74cd53-6fad-4a5c-b226-25eb178e8a53 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:18.661941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1051/2116] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:18 2022] PUT /identity/v3/projects/8325bc6ee4d3473e969b8c06de699f50/users/be4b02d5ae954712a5924bd9c2ad3555/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:18.667294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fb39c-d136-4589-842c-3f387606fc53 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.667630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fb39c-d136-4589-842c-3f387606fc53 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.667860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fb39c-d136-4589-842c-3f387606fc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.708239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1066/2117] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:18 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:44:18.711154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-395fb39c-d136-4589-842c-3f387606fc53 None None] MFA Rules not processed for user `be4b02d5ae954712a5924bd9c2ad3555`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:18.712159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74cd53-6fad-4a5c-b226-25eb178e8a53 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:18.714122 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-395fb39c-d136-4589-842c-3f387606fc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:18.726677 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74cd53-6fad-4a5c-b226-25eb178e8a53 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:18.824248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ba8fe-8c3c-4011-b3a7-755a9f438c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:18.824868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ba8fe-8c3c-4011-b3a7-755a9f438c9e admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.825547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ba8fe-8c3c-4011-b3a7-755a9f438c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.825547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ba8fe-8c3c-4011-b3a7-755a9f438c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.826082 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ba8fe-8c3c-4011-b3a7-755a9f438c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:18.827803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ba8fe-8c3c-4011-b3a7-755a9f438c9e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:18.846378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1067/2118] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:18.850877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ba8fe-8c3c-4011-b3a7-755a9f438c9e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:18.864879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ba8fe-8c3c-4011-b3a7-755a9f438c9e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:18.872113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1052/2119] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:44:18.875664 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf868f9-7335-45ba-9ad9-f89d1ff9895e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:18.876769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf868f9-7335-45ba-9ad9-f89d1ff9895e admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.877287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf868f9-7335-45ba-9ad9-f89d1ff9895e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.877750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf868f9-7335-45ba-9ad9-f89d1ff9895e admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.880182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf868f9-7335-45ba-9ad9-f89d1ff9895e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:18.883268 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf868f9-7335-45ba-9ad9-f89d1ff9895e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:18.883560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fb39c-d136-4589-842c-3f387606fc53 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:18.901868 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fb39c-d136-4589-842c-3f387606fc53 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:18.908151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1068/2120] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:18.913486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1db2b3-904c-400e-8abd-5f6a6146532a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:18.914065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1db2b3-904c-400e-8abd-5f6a6146532a admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.914904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1db2b3-904c-400e-8abd-5f6a6146532a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.914904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1db2b3-904c-400e-8abd-5f6a6146532a admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.915306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1db2b3-904c-400e-8abd-5f6a6146532a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:18.920139 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1db2b3-904c-400e-8abd-5f6a6146532a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:18.923624 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf868f9-7335-45ba-9ad9-f89d1ff9895e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:18.944210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf868f9-7335-45ba-9ad9-f89d1ff9895e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:18.949482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110eeddd-2350-46d9-bc0b-b4bf38df006d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:18.950174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eeddd-2350-46d9-bc0b-b4bf38df006d admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:18.950457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eeddd-2350-46d9-bc0b-b4bf38df006d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:18.951347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eeddd-2350-46d9-bc0b-b4bf38df006d admin admin] PATH_INFO: `/v3/projects/8c494b8167314a3aacd699afa7e6a509/users/7c623b10040f458a92aa9db4d96d8271/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:18.968356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110eeddd-2350-46d9-bc0b-b4bf38df006d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c494b8167314a3aacd699afa7e6a509, user_id=7c623b10040f458a92aa9db4d96d8271, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:18.971160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110eeddd-2350-46d9-bc0b-b4bf38df006d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:19.033415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1053/2121] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:18 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:19.039781 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1db2b3-904c-400e-8abd-5f6a6146532a admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:19.039781 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0e1db2b3-904c-400e-8abd-5f6a6146532a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:19.043811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1069/2122] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:18 2022] PUT /identity/v3/projects/8c494b8167314a3aacd699afa7e6a509/users/7c623b10040f458a92aa9db4d96d8271/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:19.044532 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110eeddd-2350-46d9-bc0b-b4bf38df006d admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:19.099339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:19.136848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1db2b3-904c-400e-8abd-5f6a6146532a admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:19.143501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:19.145192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:19.145655 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:19.146212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] PATH_INFO: `/v3/projects/e81ea6c4c3714416b9d51e152730d997` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:19.147557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81ea6c4c3714416b9d51e152730d997)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:19.149843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:19.172813 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110eeddd-2350-46d9-bc0b-b4bf38df006d admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:19.188135 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4290c70-9525-4024-894b-d467ac253ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:19.190609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4290c70-9525-4024-894b-d467ac253ead admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:19.193687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4290c70-9525-4024-894b-d467ac253ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:19.194004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4290c70-9525-4024-894b-d467ac253ead admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:19.194664 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4290c70-9525-4024-894b-d467ac253ead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:19.196640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4290c70-9525-4024-894b-d467ac253ead admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:19.278507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81ea6c4c3714416b9d51e152730d997', 'request_id': 'req-11965be0-087e-44bd-80a1-2b76b5d36f49'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:19.292700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1070/2123] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:19 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:19.292700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645a84c-91e9-4b4b-8f74-ec83781cff46 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:19.292700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645a84c-91e9-4b4b-8f74-ec83781cff46 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:19.292700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645a84c-91e9-4b4b-8f74-ec83781cff46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:19.302436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81ea6c4c3714416b9d51e152730d997', 'request_id': 'req-11965be0-087e-44bd-80a1-2b76b5d36f49'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:19.308582 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:19.315114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81ea6c4c3714416b9d51e152730d997', 'request_id': 'req-11965be0-087e-44bd-80a1-2b76b5d36f49'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:19.321725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c645a84c-91e9-4b4b-8f74-ec83781cff46 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:19.323636 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c645a84c-91e9-4b4b-8f74-ec83781cff46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:19.400627 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] The token cache is being invalidate because project e81ea6c4c3714416b9d51e152730d997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:19.401031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:19.401724 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:19.403857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1071/2124] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:19.407101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1054/2125] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:19 2022] DELETE /identity/v3/projects/e81ea6c4c3714416b9d51e152730d997 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:19.407973 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:19.409328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:19.414805 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67070992-2402-4f79-9a81-5c453e0cfb7c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:19.415326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67070992-2402-4f79-9a81-5c453e0cfb7c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:19.415686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67070992-2402-4f79-9a81-5c453e0cfb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:19.424510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11965be0-087e-44bd-80a1-2b76b5d36f49 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:19.434491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47327800-36b2-4497-a2ae-eec675e11f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:19.434491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47327800-36b2-4497-a2ae-eec675e11f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:19.434491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47327800-36b2-4497-a2ae-eec675e11f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:19.434491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47327800-36b2-4497-a2ae-eec675e11f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:19.434804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47327800-36b2-4497-a2ae-eec675e11f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:19.434804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47327800-36b2-4497-a2ae-eec675e11f2a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:19.445462 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1055/2126] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:19.446432 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-67070992-2402-4f79-9a81-5c453e0cfb7c None None] MFA Rules not processed for user `5b3dc71b4413469abf80a33687bc927f`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:19.446586 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47327800-36b2-4497-a2ae-eec675e11f2a admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:19.447690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-67070992-2402-4f79-9a81-5c453e0cfb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:19.448348 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-47327800-36b2-4497-a2ae-eec675e11f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:19.462439 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47327800-36b2-4497-a2ae-eec675e11f2a admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:19.467910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:19.468693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:19.468974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:19.469244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] PATH_INFO: `/v3/users/30c4eb49e6fb4807a65dee11e5b6f65e` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:19.470514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c4eb49e6fb4807a65dee11e5b6f65e)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:19.472430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:19.526438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1072/2127] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:19 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:19.527372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67070992-2402-4f79-9a81-5c453e0cfb7c None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:19.528918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-67070992-2402-4f79-9a81-5c453e0cfb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:19.582346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:19.586817 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67070992-2402-4f79-9a81-5c453e0cfb7c None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:19.590874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da496c5-b5da-435c-ba76-f14757b14523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:19.591543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da496c5-b5da-435c-ba76-f14757b14523 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:19.591813 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da496c5-b5da-435c-ba76-f14757b14523 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:19.603039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da496c5-b5da-435c-ba76-f14757b14523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:19.603039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0da496c5-b5da-435c-ba76-f14757b14523 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:19.604657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c4eb49e6fb4807a65dee11e5b6f65e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:19.605495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c4eb49e6fb4807a65dee11e5b6f65e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:19.606894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da496c5-b5da-435c-ba76-f14757b14523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:19.608388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da496c5-b5da-435c-ba76-f14757b14523 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:19.622545 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1073/2128] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:19.623243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da496c5-b5da-435c-ba76-f14757b14523 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:19.635135 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da496c5-b5da-435c-ba76-f14757b14523 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:19.641436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee968bc-162d-4c0d-a6d0-93cc017ff19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:19.643450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee968bc-162d-4c0d-a6d0-93cc017ff19a admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:19.643450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee968bc-162d-4c0d-a6d0-93cc017ff19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:19.643450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee968bc-162d-4c0d-a6d0-93cc017ff19a admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:19.643450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee968bc-162d-4c0d-a6d0-93cc017ff19a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:19.644832 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee968bc-162d-4c0d-a6d0-93cc017ff19a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:19.657084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1074/2129] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:19.658100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee968bc-162d-4c0d-a6d0-93cc017ff19a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:19.658671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c4eb49e6fb4807a65dee11e5b6f65e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:19.660685 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7ee968bc-162d-4c0d-a6d0-93cc017ff19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:19.668485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c4eb49e6fb4807a65dee11e5b6f65e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:19.669771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c4eb49e6fb4807a65dee11e5b6f65e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:19.673135 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee968bc-162d-4c0d-a6d0-93cc017ff19a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:19.678128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7ede9-433e-433e-aa23-14f0bf038034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:19.679576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7ede9-433e-433e-aa23-14f0bf038034 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:19.679576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7ede9-433e-433e-aa23-14f0bf038034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:19.679576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7ede9-433e-433e-aa23-14f0bf038034 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:19.679854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7ede9-433e-433e-aa23-14f0bf038034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:19.681778 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7ede9-433e-433e-aa23-14f0bf038034 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:19.695170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1056/2130] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:19 2022] DELETE /identity/v3/users/30c4eb49e6fb4807a65dee11e5b6f65e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:19.695170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:19.703235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:19.769471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1075/2131] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:19 2022] POST /identity/v3/projects => generated 403 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:19.774594 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7ede9-433e-433e-aa23-14f0bf038034 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:19.777547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-edf7ede9-433e-433e-aa23-14f0bf038034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:19.779521 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb4901-5d14-4fb4-be31-54a220fdbabc admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:19.875657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96adeb0-5ff5-411a-a15b-3d19d1223223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:19.876453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7ede9-433e-433e-aa23-14f0bf038034 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:19.878074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96adeb0-5ff5-411a-a15b-3d19d1223223 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:19.878702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96adeb0-5ff5-411a-a15b-3d19d1223223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:19.880816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96adeb0-5ff5-411a-a15b-3d19d1223223 admin admin] PATH_INFO: `/v3/projects/88e85c6937b9477dbaea5664ea918bb4/users/4a0c2525ff6b4127b3c3873baaf80452/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:19.882487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3a947-ccbd-48e4-add1-3720f0f8a114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:19.883198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3a947-ccbd-48e4-add1-3720f0f8a114 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:19.883692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3a947-ccbd-48e4-add1-3720f0f8a114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:19.884064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3a947-ccbd-48e4-add1-3720f0f8a114 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:19.884812 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3a947-ccbd-48e4-add1-3720f0f8a114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:19.887045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3a947-ccbd-48e4-add1-3720f0f8a114 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:19.901947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96adeb0-5ff5-411a-a15b-3d19d1223223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e85c6937b9477dbaea5664ea918bb4, user_id=4a0c2525ff6b4127b3c3873baaf80452, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:19.905124 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96adeb0-5ff5-411a-a15b-3d19d1223223 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:19.918133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:19.984455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1057/2132] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:19 2022] PUT /identity/v3/projects/88e85c6937b9477dbaea5664ea918bb4/users/4a0c2525ff6b4127b3c3873baaf80452/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:19.988508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1076/2133] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:19 2022] POST /identity/v3/users => generated 308 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:19.992010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96adeb0-5ff5-411a-a15b-3d19d1223223 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:19.996255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd1a61-9cdd-483b-b96e-941e706d698f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:19.997431 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd1a61-9cdd-483b-b96e-941e706d698f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:19.999252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd1a61-9cdd-483b-b96e-941e706d698f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.050322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-65fd1a61-9cdd-483b-b96e-941e706d698f None None] MFA Rules not processed for user `4a0c2525ff6b4127b3c3873baaf80452`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:20.054517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-65fd1a61-9cdd-483b-b96e-941e706d698f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:20.119883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96adeb0-5ff5-411a-a15b-3d19d1223223 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:20.134718 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336c468-fb96-40c6-8540-976c580111e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:20.135726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336c468-fb96-40c6-8540-976c580111e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:20.136005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336c468-fb96-40c6-8540-976c580111e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:20.136270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336c468-fb96-40c6-8540-976c580111e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.136973 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336c468-fb96-40c6-8540-976c580111e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:20.138658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336c468-fb96-40c6-8540-976c580111e5 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:20.154063 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1058/2134] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:20.160838 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336c468-fb96-40c6-8540-976c580111e5 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:20.171892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1077/2135] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:20.173326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336c468-fb96-40c6-8540-976c580111e5 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:20.176811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd1a61-9cdd-483b-b96e-941e706d698f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:20.181963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc7c79-4e7f-46d7-b7a4-05e74c29ae7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:20.182638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc7c79-4e7f-46d7-b7a4-05e74c29ae7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:20.184111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc7c79-4e7f-46d7-b7a4-05e74c29ae7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:20.184111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc7c79-4e7f-46d7-b7a4-05e74c29ae7d admin admin] PATH_INFO: `/v3/projects/6602f57678cf4a99aeeb3f36acdb0daa/users/40c7d434133d431b85fe1c1a13b3b358/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.199346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd1a61-9cdd-483b-b96e-941e706d698f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:20.204290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc7c79-4e7f-46d7-b7a4-05e74c29ae7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6602f57678cf4a99aeeb3f36acdb0daa, user_id=40c7d434133d431b85fe1c1a13b3b358, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:20.205031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03358fa-7cd4-4317-9cce-12c5e7b8130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:20.205659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03358fa-7cd4-4317-9cce-12c5e7b8130b admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:20.206502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03358fa-7cd4-4317-9cce-12c5e7b8130b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:20.206502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03358fa-7cd4-4317-9cce-12c5e7b8130b admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.207975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03358fa-7cd4-4317-9cce-12c5e7b8130b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:20.208176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc7c79-4e7f-46d7-b7a4-05e74c29ae7d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:20.211607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03358fa-7cd4-4317-9cce-12c5e7b8130b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:20.263385 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1059/2136] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:20 2022] PUT /identity/v3/projects/6602f57678cf4a99aeeb3f36acdb0daa/users/40c7d434133d431b85fe1c1a13b3b358/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:20.267030 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc7c79-4e7f-46d7-b7a4-05e74c29ae7d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:20.269085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c9cc7c79-4e7f-46d7-b7a4-05e74c29ae7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:20.341443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc7c79-4e7f-46d7-b7a4-05e74c29ae7d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:20.342326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1078/2137] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:20 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:20.343647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03358fa-7cd4-4317-9cce-12c5e7b8130b admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:20.373718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:20.415455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:20.416294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:20.417000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:20.417246 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] PATH_INFO: `/v3/projects/c709b3977940497bbd8f04a337cdbb96` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.418446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c709b3977940497bbd8f04a337cdbb96)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:20.420359 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:20.434077 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03358fa-7cd4-4317-9cce-12c5e7b8130b admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:20.439530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f6119-50a1-4aad-9c61-2ff1489fc131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:20.440143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f6119-50a1-4aad-9c61-2ff1489fc131 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:20.440460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f6119-50a1-4aad-9c61-2ff1489fc131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:20.440727 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f6119-50a1-4aad-9c61-2ff1489fc131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.441370 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f6119-50a1-4aad-9c61-2ff1489fc131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:20.443607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f6119-50a1-4aad-9c61-2ff1489fc131 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:20.460599 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1079/2138] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:20.460599 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f6119-50a1-4aad-9c61-2ff1489fc131 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:20.468689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f6119-50a1-4aad-9c61-2ff1489fc131 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:20.473349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14d193-fcb3-4e9a-b234-a6704f6bf732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:20.474017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14d193-fcb3-4e9a-b234-a6704f6bf732 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:20.474274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14d193-fcb3-4e9a-b234-a6704f6bf732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:20.474514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14d193-fcb3-4e9a-b234-a6704f6bf732 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.475449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14d193-fcb3-4e9a-b234-a6704f6bf732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:20.477042 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14d193-fcb3-4e9a-b234-a6704f6bf732 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:20.555735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c709b3977940497bbd8f04a337cdbb96', 'request_id': 'req-e7132492-750a-42c2-8043-405b5a08d93e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:20.561531 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c709b3977940497bbd8f04a337cdbb96', 'request_id': 'req-e7132492-750a-42c2-8043-405b5a08d93e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:20.568800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c709b3977940497bbd8f04a337cdbb96', 'request_id': 'req-e7132492-750a-42c2-8043-405b5a08d93e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:20.590532 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1080/2139] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:20 2022] POST /identity/v3/users => generated 299 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:20.591483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14d193-fcb3-4e9a-b234-a6704f6bf732 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:20.601606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14d193-fcb3-4e9a-b234-a6704f6bf732 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:20.646474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] The token cache is being invalidate because project c709b3977940497bbd8f04a337cdbb96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:20.646777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:20.647393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:20.649012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1060/2140] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:20 2022] DELETE /identity/v3/projects/c709b3977940497bbd8f04a337cdbb96 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:20.649748 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:20.651107 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:20.659728 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ce7ac-47eb-4c11-a271-092775dfac13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:20.660401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ce7ac-47eb-4c11-a271-092775dfac13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:20.661214 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ce7ac-47eb-4c11-a271-092775dfac13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:20.661486 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ce7ac-47eb-4c11-a271-092775dfac13 admin admin] PATH_INFO: `/v3/projects/6602f57678cf4a99aeeb3f36acdb0daa/users/40c7d434133d431b85fe1c1a13b3b358/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.664203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ce7ac-47eb-4c11-a271-092775dfac13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6602f57678cf4a99aeeb3f36acdb0daa, user_id=40c7d434133d431b85fe1c1a13b3b358, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:20.669395 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ce7ac-47eb-4c11-a271-092775dfac13 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:20.680635 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1081/2141] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:20 2022] PUT /identity/v3/projects/6602f57678cf4a99aeeb3f36acdb0daa/users/40c7d434133d431b85fe1c1a13b3b358/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:20.681930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bc13b-901d-49b8-afcf-8381656ecf50 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:20.682211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bc13b-901d-49b8-afcf-8381656ecf50 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:20.682480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bc13b-901d-49b8-afcf-8381656ecf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.725995 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-fd8bc13b-901d-49b8-afcf-8381656ecf50 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:20.727517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-fd8bc13b-901d-49b8-afcf-8381656ecf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:20.746382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7132492-750a-42c2-8043-405b5a08d93e admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:20.757697 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:20.804902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1082/2142] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:20.807382 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894402f1-759e-4b8d-b678-5593942d6fe0 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:20.807610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894402f1-759e-4b8d-b678-5593942d6fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:20.807885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894402f1-759e-4b8d-b678-5593942d6fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.814222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fe4b0-ff91-485d-8d2e-47b99d67b5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:20.814741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fe4b0-ff91-485d-8d2e-47b99d67b5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:20.815018 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fe4b0-ff91-485d-8d2e-47b99d67b5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:20.815333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fe4b0-ff91-485d-8d2e-47b99d67b5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.815972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fe4b0-ff91-485d-8d2e-47b99d67b5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:20.817729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fe4b0-ff91-485d-8d2e-47b99d67b5e6 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:20.828121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1061/2143] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:20.832892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fe4b0-ff91-485d-8d2e-47b99d67b5e6 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:20.835128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-894402f1-759e-4b8d-b678-5593942d6fe0 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:20.836333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-894402f1-759e-4b8d-b678-5593942d6fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:20.842813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fe4b0-ff91-485d-8d2e-47b99d67b5e6 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:20.846701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1083/2144] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:44:20.848234 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66558d27-4f81-4bec-82bf-d754c24514fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:20.848897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66558d27-4f81-4bec-82bf-d754c24514fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:20.849162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66558d27-4f81-4bec-82bf-d754c24514fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:20.849424 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66558d27-4f81-4bec-82bf-d754c24514fa admin admin] PATH_INFO: `/v3/projects/6fe5d9b0365e4bfa95b9e4f05d5fa57a/users/0e330a2ddf0547f9a0b930f4797a7d66/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.850929 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894402f1-759e-4b8d-b678-5593942d6fe0 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:20.862143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894402f1-759e-4b8d-b678-5593942d6fe0 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:20.866028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66558d27-4f81-4bec-82bf-d754c24514fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe5d9b0365e4bfa95b9e4f05d5fa57a, user_id=0e330a2ddf0547f9a0b930f4797a7d66, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:20.868449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06364b0-5c70-4775-87d1-03b297583bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:20.868666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66558d27-4f81-4bec-82bf-d754c24514fa admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:20.869071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06364b0-5c70-4775-87d1-03b297583bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:20.869349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06364b0-5c70-4775-87d1-03b297583bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:20.869619 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06364b0-5c70-4775-87d1-03b297583bf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.870308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06364b0-5c70-4775-87d1-03b297583bf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:44:20.870613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06364b0-5c70-4775-87d1-03b297583bf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:20.872202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06364b0-5c70-4775-87d1-03b297583bf5 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:20.885852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1084/2145] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:44:20.889457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06364b0-5c70-4775-87d1-03b297583bf5 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:20.899764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06364b0-5c70-4775-87d1-03b297583bf5 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:20.905072 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44d09f-ca12-4614-9dfb-00053d66df46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:20.905795 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44d09f-ca12-4614-9dfb-00053d66df46 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:20.906115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44d09f-ca12-4614-9dfb-00053d66df46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:20.906387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44d09f-ca12-4614-9dfb-00053d66df46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:20.907491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44d09f-ca12-4614-9dfb-00053d66df46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:20.908989 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44d09f-ca12-4614-9dfb-00053d66df46 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:20.949994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:20.957859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1062/2146] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:20 2022] PUT /identity/v3/projects/6fe5d9b0365e4bfa95b9e4f05d5fa57a/users/0e330a2ddf0547f9a0b930f4797a7d66/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:20.960065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66558d27-4f81-4bec-82bf-d754c24514fa admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:21.028768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66558d27-4f81-4bec-82bf-d754c24514fa admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:21.034725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bfe02-81c0-4e87-9434-4578c44419ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:21.035669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bfe02-81c0-4e87-9434-4578c44419ad admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:21.036199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bfe02-81c0-4e87-9434-4578c44419ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:21.036652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3bfe02-81c0-4e87-9434-4578c44419ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:21.037621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bfe02-81c0-4e87-9434-4578c44419ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:21.043061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1085/2147] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:20 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:44:21.045032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44d09f-ca12-4614-9dfb-00053d66df46 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:21.045677 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3bfe02-81c0-4e87-9434-4578c44419ad admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:21.063290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1063/2148] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:21.064588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bfe02-81c0-4e87-9434-4578c44419ad admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:21.080012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3bfe02-81c0-4e87-9434-4578c44419ad admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:21.127509 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44d09f-ca12-4614-9dfb-00053d66df46 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:21.134494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34ee03-dd9d-4dde-b7a3-a4f27e624a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:21.135327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34ee03-dd9d-4dde-b7a3-a4f27e624a33 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:21.135822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34ee03-dd9d-4dde-b7a3-a4f27e624a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:21.136489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34ee03-dd9d-4dde-b7a3-a4f27e624a33 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:21.137311 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34ee03-dd9d-4dde-b7a3-a4f27e624a33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:21.139762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34ee03-dd9d-4dde-b7a3-a4f27e624a33 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:21.158269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd96dc4-eaf8-416c-bdb1-333098a966fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:21.159054 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd96dc4-eaf8-416c-bdb1-333098a966fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:21.159298 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd96dc4-eaf8-416c-bdb1-333098a966fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:21.159558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd96dc4-eaf8-416c-bdb1-333098a966fb admin admin] PATH_INFO: `/v3/projects/6fe5d9b0365e4bfa95b9e4f05d5fa57a/users/0e330a2ddf0547f9a0b930f4797a7d66/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:21.162465 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd96dc4-eaf8-416c-bdb1-333098a966fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe5d9b0365e4bfa95b9e4f05d5fa57a, user_id=0e330a2ddf0547f9a0b930f4797a7d66, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:21.165246 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd96dc4-eaf8-416c-bdb1-333098a966fb admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:21.177310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1064/2149] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:21 2022] PUT /identity/v3/projects/6fe5d9b0365e4bfa95b9e4f05d5fa57a/users/0e330a2ddf0547f9a0b930f4797a7d66/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:21.188763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7b11b-8ff7-4350-8c2d-c900cb6b02ee None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:21.189055 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7b11b-8ff7-4350-8c2d-c900cb6b02ee None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:21.189298 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7b11b-8ff7-4350-8c2d-c900cb6b02ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:21.221634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-39d7b11b-8ff7-4350-8c2d-c900cb6b02ee None None] MFA Rules not processed for user `0e330a2ddf0547f9a0b930f4797a7d66`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:21.223900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-39d7b11b-8ff7-4350-8c2d-c900cb6b02ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:21.268788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1086/2150] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:21 2022] POST /identity/v3/users => generated 283 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:44:21.270737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34ee03-dd9d-4dde-b7a3-a4f27e624a33 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:21.282296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:21.286339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34ee03-dd9d-4dde-b7a3-a4f27e624a33 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:21.331707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1065/2151] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:44:21.339826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7b11b-8ff7-4350-8c2d-c900cb6b02ee None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:21.342505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-39d7b11b-8ff7-4350-8c2d-c900cb6b02ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:21.368978 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba90cd9-a773-46c2-b996-537d345dcc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:21.369968 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba90cd9-a773-46c2-b996-537d345dcc62 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:21.370431 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba90cd9-a773-46c2-b996-537d345dcc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:21.371687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba90cd9-a773-46c2-b996-537d345dcc62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:21.372628 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba90cd9-a773-46c2-b996-537d345dcc62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:21.384948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba90cd9-a773-46c2-b996-537d345dcc62 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:21.407735 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1087/2152] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:44:21.412927 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba90cd9-a773-46c2-b996-537d345dcc62 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:21.435263 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba90cd9-a773-46c2-b996-537d345dcc62 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:21.441617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621d225-ca02-4cea-80e6-4885ee1c7880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:21.443071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621d225-ca02-4cea-80e6-4885ee1c7880 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:21.443657 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621d225-ca02-4cea-80e6-4885ee1c7880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:21.444157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621d225-ca02-4cea-80e6-4885ee1c7880 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:21.445012 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621d225-ca02-4cea-80e6-4885ee1c7880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:21.446986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621d225-ca02-4cea-80e6-4885ee1c7880 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:21.461017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1088/2153] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:21.465500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621d225-ca02-4cea-80e6-4885ee1c7880 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:21.471535 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7b11b-8ff7-4350-8c2d-c900cb6b02ee None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:21.478278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738e861-700b-4ca7-8121-7f42be332055 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:21.480768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738e861-700b-4ca7-8121-7f42be332055 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:21.481043 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738e861-700b-4ca7-8121-7f42be332055 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:21.481307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738e861-700b-4ca7-8121-7f42be332055 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:21.482135 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621d225-ca02-4cea-80e6-4885ee1c7880 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:21.484664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f738e861-700b-4ca7-8121-7f42be332055 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:21.492931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a7ff7-862d-4dc9-9f81-5476a34d1dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:21.493588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a7ff7-862d-4dc9-9f81-5476a34d1dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:21.493874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a7ff7-862d-4dc9-9f81-5476a34d1dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:21.494144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a7ff7-862d-4dc9-9f81-5476a34d1dcf admin admin] PATH_INFO: `/v3/projects/1c3b04ba9b2844508592bcf1aefd7859/users/3ab4610009394582a3ebdb97164e6054/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:21.532228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a7ff7-862d-4dc9-9f81-5476a34d1dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3b04ba9b2844508592bcf1aefd7859, user_id=3ab4610009394582a3ebdb97164e6054, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:21.535043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a7ff7-862d-4dc9-9f81-5476a34d1dcf admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:21.590850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738e861-700b-4ca7-8121-7f42be332055 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:21.592637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738e861-700b-4ca7-8121-7f42be332055 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:21.603136 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1066/2154] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:44:21.637860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1089/2155] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:21 2022] PUT /identity/v3/projects/1c3b04ba9b2844508592bcf1aefd7859/users/3ab4610009394582a3ebdb97164e6054/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:21.699705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278fcca-217d-43e0-9b34-8fc2f114572c None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:21.700421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278fcca-217d-43e0-9b34-8fc2f114572c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:21.700935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c278fcca-217d-43e0-9b34-8fc2f114572c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:21.752294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c278fcca-217d-43e0-9b34-8fc2f114572c None None] MFA Rules not processed for user `3ab4610009394582a3ebdb97164e6054`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:21.754904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c278fcca-217d-43e0-9b34-8fc2f114572c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:21.884377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1067/2156] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:21.897650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a7ff7-862d-4dc9-9f81-5476a34d1dcf admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:21.900216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-6c6a7ff7-862d-4dc9-9f81-5476a34d1dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:21.952922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:21.970741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a7ff7-862d-4dc9-9f81-5476a34d1dcf admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:21.975224 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14af85-96db-4454-a02b-a0477658c4f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:21.975972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14af85-96db-4454-a02b-a0477658c4f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:21.976245 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14af85-96db-4454-a02b-a0477658c4f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:21.976504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14af85-96db-4454-a02b-a0477658c4f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:21.984986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14af85-96db-4454-a02b-a0477658c4f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:21.986471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14af85-96db-4454-a02b-a0477658c4f1 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:21.996384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1090/2157] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:22.345020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943e280-5225-49b1-b711-e4a6b1b4cb72 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:22.345685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943e280-5225-49b1-b711-e4a6b1b4cb72 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:22.346045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943e280-5225-49b1-b711-e4a6b1b4cb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:22.388284 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-1943e280-5225-49b1-b711-e4a6b1b4cb72 None None] MFA Rules not processed for user `40c7d434133d431b85fe1c1a13b3b358`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:22.389657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1943e280-5225-49b1-b711-e4a6b1b4cb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:22.512512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:22.531392 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1068/2158] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:22 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:23.844414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14af85-96db-4454-a02b-a0477658c4f1 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:23.859794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14af85-96db-4454-a02b-a0477658c4f1 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:23.865690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d15b48-bb8d-4874-aa1e-7f51521350b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:23.866373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d15b48-bb8d-4874-aa1e-7f51521350b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:23.866648 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d15b48-bb8d-4874-aa1e-7f51521350b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:23.867163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d15b48-bb8d-4874-aa1e-7f51521350b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:23.870862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-06d15b48-bb8d-4874-aa1e-7f51521350b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:23.923415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943e280-5225-49b1-b711-e4a6b1b4cb72 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:23.929049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1943e280-5225-49b1-b711-e4a6b1b4cb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:23.956510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d15b48-bb8d-4874-aa1e-7f51521350b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:23.958659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d15b48-bb8d-4874-aa1e-7f51521350b7 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:23.970812 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1091/2159] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:23.992256 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:24.028880 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943e280-5225-49b1-b711-e4a6b1b4cb72 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:24.034995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:24.035899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:24.036176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:24.036444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] PATH_INFO: `/v3/users/4b5b28da1547489f95960e2ebaddd7fb` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:24.038167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b5b28da1547489f95960e2ebaddd7fb)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:24.040656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:24.287637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b5b28da1547489f95960e2ebaddd7fb'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:24.288634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b5b28da1547489f95960e2ebaddd7fb'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:24.393008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b5b28da1547489f95960e2ebaddd7fb'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:24.412333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b5b28da1547489f95960e2ebaddd7fb'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:24.413161 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b5b28da1547489f95960e2ebaddd7fb'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:24.452779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1069/2160] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:23 2022] DELETE /identity/v3/users/4b5b28da1547489f95960e2ebaddd7fb => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:24.457383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443f482-0ea8-4b91-86e2-d79d68e4d3d4 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:24.457986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443f482-0ea8-4b91-86e2-d79d68e4d3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:24.458416 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443f482-0ea8-4b91-86e2-d79d68e4d3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:24.482659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-e443f482-0ea8-4b91-86e2-d79d68e4d3d4 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:24.484792 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e443f482-0ea8-4b91-86e2-d79d68e4d3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:24.575024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:24.578941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:24.626369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1092/2161] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:24.644232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443f482-0ea8-4b91-86e2-d79d68e4d3d4 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:24.646604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e443f482-0ea8-4b91-86e2-d79d68e4d3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:24.742103 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685bb1e-6c8e-4e7c-97de-69e165a9af20 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:24.748395 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:24.749065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:24.749378 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:24.749580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] PATH_INFO: `/v3/users/5b3dc71b4413469abf80a33687bc927f` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:24.750798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b3dc71b4413469abf80a33687bc927f)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:24.752766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:24.760734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443f482-0ea8-4b91-86e2-d79d68e4d3d4 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:24.767229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fc738-2d77-4936-9429-b2c705bf08f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:24.768110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fc738-2d77-4936-9429-b2c705bf08f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:24.768646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fc738-2d77-4936-9429-b2c705bf08f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:24.769117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fc738-2d77-4936-9429-b2c705bf08f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:24.786791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6fc738-2d77-4936-9429-b2c705bf08f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:24.789292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:24.790202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6fc738-2d77-4936-9429-b2c705bf08f6 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:24.806522 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1093/2162] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:44:25.048301 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b3dc71b4413469abf80a33687bc927f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:25.049312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b3dc71b4413469abf80a33687bc927f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:25.135130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b3dc71b4413469abf80a33687bc927f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:25.142212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b3dc71b4413469abf80a33687bc927f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:25.142953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b3dc71b4413469abf80a33687bc927f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:25.163867 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1070/2163] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:24 2022] DELETE /identity/v3/users/5b3dc71b4413469abf80a33687bc927f => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:25.526282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fc738-2d77-4936-9429-b2c705bf08f6 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:25.548898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6fc738-2d77-4936-9429-b2c705bf08f6 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:25.561494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:25.563581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:25.581420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:25.686670 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:25.688226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:25.688898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:25.689404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] PATH_INFO: `/v3/projects/67626fcccd8a44e4af89f3193ae20bf7` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:25.691332 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=67626fcccd8a44e4af89f3193ae20bf7)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:25.695717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:25.700397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a0120-b8c7-4804-857d-18522a766256 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:25.718119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:25.719039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:25.719647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:25.720133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] PATH_INFO: `/v3/projects/2d89086b8d50424485a5a8c28ab285a8` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:25.726726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d89086b8d50424485a5a8c28ab285a8)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:25.728675 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:25.849303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67626fcccd8a44e4af89f3193ae20bf7', 'request_id': 'req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:25.869138 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67626fcccd8a44e4af89f3193ae20bf7', 'request_id': 'req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:25.895424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67626fcccd8a44e4af89f3193ae20bf7', 'request_id': 'req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:25.901490 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d89086b8d50424485a5a8c28ab285a8', 'request_id': 'req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:25.909879 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d89086b8d50424485a5a8c28ab285a8', 'request_id': 'req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:25.918252 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d89086b8d50424485a5a8c28ab285a8', 'request_id': 'req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:25.982955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] The token cache is being invalidate because project 67626fcccd8a44e4af89f3193ae20bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:25.983243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:25.984019 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:25.985559 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1094/2164] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:25 2022] DELETE /identity/v3/projects/67626fcccd8a44e4af89f3193ae20bf7 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:25.989616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:25.991534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:26.036335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] The token cache is being invalidate because project 2d89086b8d50424485a5a8c28ab285a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:26.036634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:26.037475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:26.039855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1071/2165] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:25 2022] DELETE /identity/v3/projects/2d89086b8d50424485a5a8c28ab285a8 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:26.041079 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:26.043295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:26.091847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952f5a9-1b6e-41c2-a258-3244a8dc1f9a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:26.140269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2610c-cd80-43c2-b2ed-1ab3090c9c03 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:26.146028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d2d8c-da83-4589-87a8-270ff11d24f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:26.146739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d2d8c-da83-4589-87a8-270ff11d24f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:26.147016 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d2d8c-da83-4589-87a8-270ff11d24f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:26.147287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d2d8c-da83-4589-87a8-270ff11d24f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:26.149120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3e8d2d8c-da83-4589-87a8-270ff11d24f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:26.180657 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:26.181768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:26.181989 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:26.185132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] PATH_INFO: `/v3/users/40c7d434133d431b85fe1c1a13b3b358` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:26.185132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40c7d434133d431b85fe1c1a13b3b358)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:26.185915 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:26.215754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:26.234322 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d2d8c-da83-4589-87a8-270ff11d24f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:26.236558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d2d8c-da83-4589-87a8-270ff11d24f2 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:26.257989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1072/2166] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:26.260118 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf4f94-d164-4ffc-82ba-9434ded86580 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:26.260416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf4f94-d164-4ffc-82ba-9434ded86580 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:26.260690 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf4f94-d164-4ffc-82ba-9434ded86580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:26.290743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-9cbf4f94-d164-4ffc-82ba-9434ded86580 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:26.291560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9cbf4f94-d164-4ffc-82ba-9434ded86580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:26.304728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1073/2167] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:26.323935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70247e97-9551-4e8e-b57f-9a7405e3ed87 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:26.336762 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70247e97-9551-4e8e-b57f-9a7405e3ed87 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:26.337222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70247e97-9551-4e8e-b57f-9a7405e3ed87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:26.365133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-70247e97-9551-4e8e-b57f-9a7405e3ed87 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:26.366387 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-70247e97-9551-4e8e-b57f-9a7405e3ed87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:26.382062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1074/2168] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:26.386389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70247e97-9551-4e8e-b57f-9a7405e3ed87 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:26.396294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70247e97-9551-4e8e-b57f-9a7405e3ed87 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:26.401481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93fce4-7ee9-44bf-a916-0e89e087fea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:26.402230 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93fce4-7ee9-44bf-a916-0e89e087fea2 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:26.402554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93fce4-7ee9-44bf-a916-0e89e087fea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:26.403622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93fce4-7ee9-44bf-a916-0e89e087fea2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:26.403847 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93fce4-7ee9-44bf-a916-0e89e087fea2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:44:26.404210 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93fce4-7ee9-44bf-a916-0e89e087fea2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:26.405801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae93fce4-7ee9-44bf-a916-0e89e087fea2 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:26.423650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40c7d434133d431b85fe1c1a13b3b358'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:26.424521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40c7d434133d431b85fe1c1a13b3b358'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:26.428824 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1075/2169] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:44:26.433054 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93fce4-7ee9-44bf-a916-0e89e087fea2 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:26.447751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae93fce4-7ee9-44bf-a916-0e89e087fea2 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:26.458309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07b750-747f-4265-8645-427ce399c0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:26.458979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07b750-747f-4265-8645-427ce399c0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:26.459253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07b750-747f-4265-8645-427ce399c0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:26.459514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07b750-747f-4265-8645-427ce399c0f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:26.460382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07b750-747f-4265-8645-427ce399c0f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:26.463500 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07b750-747f-4265-8645-427ce399c0f3 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:26.521681 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40c7d434133d431b85fe1c1a13b3b358'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:26.522861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40c7d434133d431b85fe1c1a13b3b358'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:26.532716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40c7d434133d431b85fe1c1a13b3b358'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:26.566730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1095/2170] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:25 2022] DELETE /identity/v3/users/40c7d434133d431b85fe1c1a13b3b358 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:26.575111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1076/2171] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:26 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:44:26.580821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:26.584751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:26.715816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:26.749271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85af7c0-dbfd-4705-8075-e438eeefda51 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:26.854136 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b2446-9b0c-493c-a89b-bca0fd509e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:26.854791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b2446-9b0c-493c-a89b-bca0fd509e2f admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:26.855717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b2446-9b0c-493c-a89b-bca0fd509e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:26.855717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b2446-9b0c-493c-a89b-bca0fd509e2f admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:26.856266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b2446-9b0c-493c-a89b-bca0fd509e2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:26.859030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b2446-9b0c-493c-a89b-bca0fd509e2f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:26.938561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07b750-747f-4265-8645-427ce399c0f3 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:26.940718 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0e07b750-747f-4265-8645-427ce399c0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:26.979363 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07b750-747f-4265-8645-427ce399c0f3 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:26.986341 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f5ca1-2aff-4861-bb12-d8f06b671855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:26.987000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f5ca1-2aff-4861-bb12-d8f06b671855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:26.987274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f5ca1-2aff-4861-bb12-d8f06b671855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:26.987662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f5ca1-2aff-4861-bb12-d8f06b671855 admin admin] PATH_INFO: `/v3/projects/6602f57678cf4a99aeeb3f36acdb0daa` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:26.989022 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f5ca1-2aff-4861-bb12-d8f06b671855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6602f57678cf4a99aeeb3f36acdb0daa)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:26.991223 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f5ca1-2aff-4861-bb12-d8f06b671855 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:27.014239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1096/2172] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:26 2022] POST /identity/v3/users => generated 291 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:27.021362 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b2446-9b0c-493c-a89b-bca0fd509e2f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:27.059650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b2446-9b0c-493c-a89b-bca0fd509e2f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:27.087112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df58726-5684-432d-8a42-a5c0a52224d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:27.089034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df58726-5684-432d-8a42-a5c0a52224d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:27.091421 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df58726-5684-432d-8a42-a5c0a52224d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:27.098661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df58726-5684-432d-8a42-a5c0a52224d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:27.098661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df58726-5684-432d-8a42-a5c0a52224d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:27.098661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df58726-5684-432d-8a42-a5c0a52224d9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:27.138219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1097/2173] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:27.140796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df58726-5684-432d-8a42-a5c0a52224d9 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:27.161189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df58726-5684-432d-8a42-a5c0a52224d9 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:27.163886 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-521f5ca1-2aff-4861-bb12-d8f06b671855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6602f57678cf4a99aeeb3f36acdb0daa', 'request_id': 'req-521f5ca1-2aff-4861-bb12-d8f06b671855'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:27.175622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-521f5ca1-2aff-4861-bb12-d8f06b671855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6602f57678cf4a99aeeb3f36acdb0daa', 'request_id': 'req-521f5ca1-2aff-4861-bb12-d8f06b671855'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:27.190552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-521f5ca1-2aff-4861-bb12-d8f06b671855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6602f57678cf4a99aeeb3f36acdb0daa', 'request_id': 'req-521f5ca1-2aff-4861-bb12-d8f06b671855'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:27.268572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb65c8-7737-480f-98ce-2f98263d4b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:27.271319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb65c8-7737-480f-98ce-2f98263d4b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:27.271596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb65c8-7737-480f-98ce-2f98263d4b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:27.271856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb65c8-7737-480f-98ce-2f98263d4b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:27.273156 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb65c8-7737-480f-98ce-2f98263d4b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:27.275939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb65c8-7737-480f-98ce-2f98263d4b89 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:27.296365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1098/2174] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:27.302105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb65c8-7737-480f-98ce-2f98263d4b89 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:27.305376 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-521f5ca1-2aff-4861-bb12-d8f06b671855 admin admin] The token cache is being invalidate because project 6602f57678cf4a99aeeb3f36acdb0daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:27.313130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-521f5ca1-2aff-4861-bb12-d8f06b671855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:27.313130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-521f5ca1-2aff-4861-bb12-d8f06b671855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:27.313130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1077/2175] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:26 2022] DELETE /identity/v3/projects/6602f57678cf4a99aeeb3f36acdb0daa => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:27.315243 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b859b3-807a-4c2f-8353-31d5bf778cde None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:27.315538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b859b3-807a-4c2f-8353-31d5bf778cde None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:27.316339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b859b3-807a-4c2f-8353-31d5bf778cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:27.322799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb65c8-7737-480f-98ce-2f98263d4b89 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:27.330188 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d54bc8-a745-46e2-8d80-eb198b47f6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:27.331097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d54bc8-a745-46e2-8d80-eb198b47f6e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:27.331447 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d54bc8-a745-46e2-8d80-eb198b47f6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:27.331790 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d54bc8-a745-46e2-8d80-eb198b47f6e5 admin admin] PATH_INFO: `/v3/projects/04dc3f5e68e046fb9bf360d4f01a50fd/users/2e593646e79c443490b22657996b9108/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:27.346074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-11b859b3-807a-4c2f-8353-31d5bf778cde None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:27.348667 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-11b859b3-807a-4c2f-8353-31d5bf778cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:27.350246 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d54bc8-a745-46e2-8d80-eb198b47f6e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04dc3f5e68e046fb9bf360d4f01a50fd, user_id=2e593646e79c443490b22657996b9108, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:27.353367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d54bc8-a745-46e2-8d80-eb198b47f6e5 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:27.361446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1078/2176] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:27.370485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd508d-e813-421b-8f09-c1e7c6de170e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:27.371265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd508d-e813-421b-8f09-c1e7c6de170e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:27.371960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd508d-e813-421b-8f09-c1e7c6de170e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:27.406249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-babd508d-e813-421b-8f09-c1e7c6de170e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:27.407645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-babd508d-e813-421b-8f09-c1e7c6de170e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:27.426488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1079/2177] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:27.427683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd508d-e813-421b-8f09-c1e7c6de170e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:27.438779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd508d-e813-421b-8f09-c1e7c6de170e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:27.460797 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1099/2178] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:27 2022] PUT /identity/v3/projects/04dc3f5e68e046fb9bf360d4f01a50fd/users/2e593646e79c443490b22657996b9108/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:27.466137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabb733-b1b2-4576-80a0-3125b335b3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:27.467109 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dee436-30c4-4e74-97d5-66141ace2b20 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:27.468524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabb733-b1b2-4576-80a0-3125b335b3bc admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:27.468524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabb733-b1b2-4576-80a0-3125b335b3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:27.468524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabb733-b1b2-4576-80a0-3125b335b3bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:27.469138 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dee436-30c4-4e74-97d5-66141ace2b20 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:27.469493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dee436-30c4-4e74-97d5-66141ace2b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:27.472898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabb733-b1b2-4576-80a0-3125b335b3bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:44:27.473304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabb733-b1b2-4576-80a0-3125b335b3bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:27.475157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabb733-b1b2-4576-80a0-3125b335b3bc admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:27.494041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1080/2179] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:44:27.495056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-42dee436-30c4-4e74-97d5-66141ace2b20 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:27.496329 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-42dee436-30c4-4e74-97d5-66141ace2b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:27.500148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabb733-b1b2-4576-80a0-3125b335b3bc admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:27.513692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabb733-b1b2-4576-80a0-3125b335b3bc admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:27.562165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1100/2180] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:27.563331 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6dc78-e482-4516-9297-2b92d6a90ca9 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:27.563612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6dc78-e482-4516-9297-2b92d6a90ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:27.563884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6dc78-e482-4516-9297-2b92d6a90ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:27.576801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913a9f2-3fe6-43e5-942b-e1214648b1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:27.577391 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913a9f2-3fe6-43e5-942b-e1214648b1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:27.577665 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913a9f2-3fe6-43e5-942b-e1214648b1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:27.577969 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913a9f2-3fe6-43e5-942b-e1214648b1f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:27.578601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913a9f2-3fe6-43e5-942b-e1214648b1f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:27.580483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913a9f2-3fe6-43e5-942b-e1214648b1f2 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:27.594221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-62e6dc78-e482-4516-9297-2b92d6a90ca9 None None] MFA Rules not processed for user `7c623b10040f458a92aa9db4d96d8271`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:27.595440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-62e6dc78-e482-4516-9297-2b92d6a90ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:27.646204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1081/2181] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:27 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:44:27.646204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913a9f2-3fe6-43e5-942b-e1214648b1f2 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:27.646204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e913a9f2-3fe6-43e5-942b-e1214648b1f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:27.665096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1101/2182] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:27.665942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6dc78-e482-4516-9297-2b92d6a90ca9 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:27.675776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6dc78-e482-4516-9297-2b92d6a90ca9 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:27.714719 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913a9f2-3fe6-43e5-942b-e1214648b1f2 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:27.718510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ace9b4-887d-4885-9db8-b8dde2f38765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:27.719126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ace9b4-887d-4885-9db8-b8dde2f38765 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:27.719403 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ace9b4-887d-4885-9db8-b8dde2f38765 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:27.719677 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ace9b4-887d-4885-9db8-b8dde2f38765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:27.755164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b25b06-c548-4cb7-b510-fc5ab53523f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:27.755783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b25b06-c548-4cb7-b510-fc5ab53523f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:27.756056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b25b06-c548-4cb7-b510-fc5ab53523f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:27.756329 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b25b06-c548-4cb7-b510-fc5ab53523f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:27.756945 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b25b06-c548-4cb7-b510-fc5ab53523f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:27.758891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b25b06-c548-4cb7-b510-fc5ab53523f9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:27.785172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ace9b4-887d-4885-9db8-b8dde2f38765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:27.786700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ace9b4-887d-4885-9db8-b8dde2f38765 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:27.796364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1082/2183] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:27.797148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ace9b4-887d-4885-9db8-b8dde2f38765 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:27.798476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-16ace9b4-887d-4885-9db8-b8dde2f38765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:27.813793 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ace9b4-887d-4885-9db8-b8dde2f38765 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:27.819659 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdb6fc-4f5c-4863-91ec-0b7015dd987e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:27.820267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdb6fc-4f5c-4863-91ec-0b7015dd987e admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:27.820699 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdb6fc-4f5c-4863-91ec-0b7015dd987e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:27.821072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdb6fc-4f5c-4863-91ec-0b7015dd987e admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:27.822025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdb6fc-4f5c-4863-91ec-0b7015dd987e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:27.828178 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdb6fc-4f5c-4863-91ec-0b7015dd987e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:27.876157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1102/2184] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:27 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:27.880942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b25b06-c548-4cb7-b510-fc5ab53523f9 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:27.892976 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b25b06-c548-4cb7-b510-fc5ab53523f9 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:27.901331 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e275915-9231-46a3-9e32-711b5004000f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:27.901507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1083/2185] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:27 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:27.905404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e275915-9231-46a3-9e32-711b5004000f admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:27.905800 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e275915-9231-46a3-9e32-711b5004000f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:27.906180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e275915-9231-46a3-9e32-711b5004000f admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:27.907066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e275915-9231-46a3-9e32-711b5004000f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:27.908852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e275915-9231-46a3-9e32-711b5004000f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:27.909203 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdb6fc-4f5c-4863-91ec-0b7015dd987e admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:27.923726 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1103/2186] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:44:27.928646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e275915-9231-46a3-9e32-711b5004000f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:27.939130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e275915-9231-46a3-9e32-711b5004000f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:28.003393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdb6fc-4f5c-4863-91ec-0b7015dd987e admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:28.010254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45a0f1-eae4-46e3-be93-f29642b1f15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:28.011089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a0f1-eae4-46e3-be93-f29642b1f15b admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:28.011362 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a0f1-eae4-46e3-be93-f29642b1f15b admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:28.011632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45a0f1-eae4-46e3-be93-f29642b1f15b admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:28.012254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45a0f1-eae4-46e3-be93-f29642b1f15b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:28.014245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45a0f1-eae4-46e3-be93-f29642b1f15b admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:28.017221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590b0c-43e4-412a-ac72-b5fdad01b56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:28.017983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590b0c-43e4-412a-ac72-b5fdad01b56a admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:28.018262 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590b0c-43e4-412a-ac72-b5fdad01b56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:28.018559 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590b0c-43e4-412a-ac72-b5fdad01b56a admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:28.019204 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590b0c-43e4-412a-ac72-b5fdad01b56a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:28.021226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590b0c-43e4-412a-ac72-b5fdad01b56a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:28.032229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1104/2187] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:44:28.037070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590b0c-43e4-412a-ac72-b5fdad01b56a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:28.068128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590b0c-43e4-412a-ac72-b5fdad01b56a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:28.068128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449db9a4-9378-4d21-8c14-b767fb44cec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:28.068128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449db9a4-9378-4d21-8c14-b767fb44cec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:28.068128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449db9a4-9378-4d21-8c14-b767fb44cec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:28.071602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449db9a4-9378-4d21-8c14-b767fb44cec7 admin admin] PATH_INFO: `/v3/projects/577338b7d2b44b3e9ba41d6b1120dd8e/users/327a7f16283a49d2b4eefab257baa502/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:28.072546 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449db9a4-9378-4d21-8c14-b767fb44cec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=577338b7d2b44b3e9ba41d6b1120dd8e, user_id=327a7f16283a49d2b4eefab257baa502, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:28.076821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449db9a4-9378-4d21-8c14-b767fb44cec7 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:28.371947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1084/2188] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:27 2022] POST /identity/v3/users => generated 292 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:28.389033 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45a0f1-eae4-46e3-be93-f29642b1f15b admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:28.404545 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1105/2189] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:28 2022] PUT /identity/v3/projects/577338b7d2b44b3e9ba41d6b1120dd8e/users/327a7f16283a49d2b4eefab257baa502/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:28.424352 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e37a35-0c30-478c-a702-b762fef31cb7 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:28.424831 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e37a35-0c30-478c-a702-b762fef31cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:28.425295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e37a35-0c30-478c-a702-b762fef31cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:28.437474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45a0f1-eae4-46e3-be93-f29642b1f15b admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:28.521326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-93e37a35-0c30-478c-a702-b762fef31cb7 None None] MFA Rules not processed for user `327a7f16283a49d2b4eefab257baa502`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:28.525253 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-93e37a35-0c30-478c-a702-b762fef31cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:28.606499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c5483-a1fc-49cb-91ce-b2d640d5d4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:28.606499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c5483-a1fc-49cb-91ce-b2d640d5d4c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:28.606727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c5483-a1fc-49cb-91ce-b2d640d5d4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:28.607171 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c5483-a1fc-49cb-91ce-b2d640d5d4c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:28.608010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c5483-a1fc-49cb-91ce-b2d640d5d4c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:28.610390 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c5483-a1fc-49cb-91ce-b2d640d5d4c0 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:28.624006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1085/2190] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:28.627891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c5483-a1fc-49cb-91ce-b2d640d5d4c0 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:28.641777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c5483-a1fc-49cb-91ce-b2d640d5d4c0 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:28.649390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:28.650217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b9d37-2228-4327-a5d3-0f05016cfc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:28.651530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b9d37-2228-4327-a5d3-0f05016cfc55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:28.652004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b9d37-2228-4327-a5d3-0f05016cfc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:28.652427 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b9d37-2228-4327-a5d3-0f05016cfc55 admin admin] PATH_INFO: `/v3/projects/728e268ef22e470194545edea611eb32/users/49fb50eb99b64f588c0312c2935c82fa/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:28.678550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b9d37-2228-4327-a5d3-0f05016cfc55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=728e268ef22e470194545edea611eb32, user_id=49fb50eb99b64f588c0312c2935c82fa, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:28.678550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b9d37-2228-4327-a5d3-0f05016cfc55 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:28.715519 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1106/2191] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:28.717056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e37a35-0c30-478c-a702-b762fef31cb7 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:28.741098 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e37a35-0c30-478c-a702-b762fef31cb7 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:28.752000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbb978-fa5d-4882-98b8-abb99faaf201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:28.752734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbb978-fa5d-4882-98b8-abb99faaf201 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:28.753105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbb978-fa5d-4882-98b8-abb99faaf201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:28.753548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cbb978-fa5d-4882-98b8-abb99faaf201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:28.758817 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbb978-fa5d-4882-98b8-abb99faaf201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:28.758817 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cbb978-fa5d-4882-98b8-abb99faaf201 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:28.818380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1086/2192] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:28 2022] PUT /identity/v3/projects/728e268ef22e470194545edea611eb32/users/49fb50eb99b64f588c0312c2935c82fa/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:28.818380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b9d37-2228-4327-a5d3-0f05016cfc55 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:28.904473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1107/2193] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:28 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:28.915393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbb978-fa5d-4882-98b8-abb99faaf201 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:28.966274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b9d37-2228-4327-a5d3-0f05016cfc55 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:29.040755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb59c09-fe9e-489d-a226-95259d8be26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:29.041580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb59c09-fe9e-489d-a226-95259d8be26b admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:29.042041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb59c09-fe9e-489d-a226-95259d8be26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:29.042463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb59c09-fe9e-489d-a226-95259d8be26b admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:29.043441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb59c09-fe9e-489d-a226-95259d8be26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:29.045363 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb59c09-fe9e-489d-a226-95259d8be26b admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:29.057400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1087/2194] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:44:29.061544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb59c09-fe9e-489d-a226-95259d8be26b admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:29.070235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cbb978-fa5d-4882-98b8-abb99faaf201 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:29.071859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb59c09-fe9e-489d-a226-95259d8be26b admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:29.077091 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782df59d-a987-49cc-93ff-a8adce565187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:29.077752 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782df59d-a987-49cc-93ff-a8adce565187 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:29.078057 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782df59d-a987-49cc-93ff-a8adce565187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:29.078326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782df59d-a987-49cc-93ff-a8adce565187 admin admin] PATH_INFO: `/v3/projects/728e268ef22e470194545edea611eb32/users/49fb50eb99b64f588c0312c2935c82fa/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:29.080338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782df59d-a987-49cc-93ff-a8adce565187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=728e268ef22e470194545edea611eb32, user_id=49fb50eb99b64f588c0312c2935c82fa, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:29.083638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782df59d-a987-49cc-93ff-a8adce565187 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:29.083915 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee45af4-6b09-4389-8ebe-165425955c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:29.084703 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee45af4-6b09-4389-8ebe-165425955c7a admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:29.085079 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee45af4-6b09-4389-8ebe-165425955c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:29.085443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee45af4-6b09-4389-8ebe-165425955c7a admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:29.087432 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee45af4-6b09-4389-8ebe-165425955c7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:29.093897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee45af4-6b09-4389-8ebe-165425955c7a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:29.101646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1088/2195] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:29 2022] PUT /identity/v3/projects/728e268ef22e470194545edea611eb32/users/49fb50eb99b64f588c0312c2935c82fa/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:29.109929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:29.228185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1108/2196] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:28 2022] POST /identity/v3/users => generated 294 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:29.232530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782df59d-a987-49cc-93ff-a8adce565187 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:29.308872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782df59d-a987-49cc-93ff-a8adce565187 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:29.313547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25080b47-7d12-46e9-b67d-2ce0e842c550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:29.314155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25080b47-7d12-46e9-b67d-2ce0e842c550 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:29.314424 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25080b47-7d12-46e9-b67d-2ce0e842c550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:29.314709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25080b47-7d12-46e9-b67d-2ce0e842c550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:29.315325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25080b47-7d12-46e9-b67d-2ce0e842c550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:29.317758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25080b47-7d12-46e9-b67d-2ce0e842c550 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:29.328276 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1089/2197] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:29.332673 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee45af4-6b09-4389-8ebe-165425955c7a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:29.342700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee45af4-6b09-4389-8ebe-165425955c7a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:29.347220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdd5d1-8158-498c-b8d8-ee246917ff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:29.347895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdd5d1-8158-498c-b8d8-ee246917ff59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:29.348202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdd5d1-8158-498c-b8d8-ee246917ff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:29.348481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdd5d1-8158-498c-b8d8-ee246917ff59 admin admin] PATH_INFO: `/v3/projects/deef889de876421b96f65d14d05f6344/users/df3c3f11c4c04471a78ee5518976fbcc/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:29.364489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdd5d1-8158-498c-b8d8-ee246917ff59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deef889de876421b96f65d14d05f6344, user_id=df3c3f11c4c04471a78ee5518976fbcc, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:29.366605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdd5d1-8158-498c-b8d8-ee246917ff59 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:29.494697 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1109/2198] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:29 2022] PUT /identity/v3/projects/deef889de876421b96f65d14d05f6344/users/df3c3f11c4c04471a78ee5518976fbcc/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:29.499267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25080b47-7d12-46e9-b67d-2ce0e842c550 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:29.512748 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25080b47-7d12-46e9-b67d-2ce0e842c550 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:29.586082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b54001-0160-4138-a605-b2eaf03dbddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:29.586693 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b54001-0160-4138-a605-b2eaf03dbddd admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:29.586964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b54001-0160-4138-a605-b2eaf03dbddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:29.587231 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b54001-0160-4138-a605-b2eaf03dbddd admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:29.587936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b54001-0160-4138-a605-b2eaf03dbddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:29.589459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b54001-0160-4138-a605-b2eaf03dbddd admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:29.600311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1090/2199] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:29 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:29.604600 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdd5d1-8158-498c-b8d8-ee246917ff59 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:29.618878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdd5d1-8158-498c-b8d8-ee246917ff59 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:29.625096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbdc5b-1626-4e71-942c-0214a9f7225b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:29.625762 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbdc5b-1626-4e71-942c-0214a9f7225b admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:29.626075 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbdc5b-1626-4e71-942c-0214a9f7225b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:29.626333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbdc5b-1626-4e71-942c-0214a9f7225b admin admin] PATH_INFO: `/v3/projects/deef889de876421b96f65d14d05f6344/users/df3c3f11c4c04471a78ee5518976fbcc/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:29.634474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbdc5b-1626-4e71-942c-0214a9f7225b admin admin] RBAC: Authorizing `identity:create_grant(project_id=deef889de876421b96f65d14d05f6344, user_id=df3c3f11c4c04471a78ee5518976fbcc, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:29.635077 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbdc5b-1626-4e71-942c-0214a9f7225b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:29.647327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1110/2200] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:29 2022] PUT /identity/v3/projects/deef889de876421b96f65d14d05f6344/users/df3c3f11c4c04471a78ee5518976fbcc/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:29.665368 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09d191-0561-440c-9440-a058ff4d043a None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:29.665728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09d191-0561-440c-9440-a058ff4d043a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:29.666240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09d191-0561-440c-9440-a058ff4d043a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:29.723860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-fe09d191-0561-440c-9440-a058ff4d043a None None] MFA Rules not processed for user `df3c3f11c4c04471a78ee5518976fbcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:29.724587 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-fe09d191-0561-440c-9440-a058ff4d043a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:29.831720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539956-f9c3-4f52-979f-6d2032302611 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:29.832403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539956-f9c3-4f52-979f-6d2032302611 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:29.832403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539956-f9c3-4f52-979f-6d2032302611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:29.907789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-3c539956-f9c3-4f52-979f-6d2032302611 None None] MFA Rules not processed for user `2e593646e79c443490b22657996b9108`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:29.909076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3c539956-f9c3-4f52-979f-6d2032302611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:29.924961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1091/2201] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:44:29.964766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09d191-0561-440c-9440-a058ff4d043a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:29.969974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-fe09d191-0561-440c-9440-a058ff4d043a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:30.065351 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:30.081880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1111/2202] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:30.187463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09d191-0561-440c-9440-a058ff4d043a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:30.195248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:30.200849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4583d9ec-cf13-4252-9c18-0bb6ceb28554 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:30.201649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583d9ec-cf13-4252-9c18-0bb6ceb28554 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:30.202152 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583d9ec-cf13-4252-9c18-0bb6ceb28554 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:30.202679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4583d9ec-cf13-4252-9c18-0bb6ceb28554 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:30.282878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4583d9ec-cf13-4252-9c18-0bb6ceb28554 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:30.285946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4583d9ec-cf13-4252-9c18-0bb6ceb28554 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:30.299250 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1092/2203] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:44:30.677250 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:31.078773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c539956-f9c3-4f52-979f-6d2032302611 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:31.096026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c539956-f9c3-4f52-979f-6d2032302611 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:31.101317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e06c3-ee12-4413-bd70-26593e360f47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:31.102514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e06c3-ee12-4413-bd70-26593e360f47 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:31.102782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e06c3-ee12-4413-bd70-26593e360f47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:31.103550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e06c3-ee12-4413-bd70-26593e360f47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:31.115681 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e06c3-ee12-4413-bd70-26593e360f47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:31.118055 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e06c3-ee12-4413-bd70-26593e360f47 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:31.144951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1112/2204] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:31.232565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:31.898434 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:32.526881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:33.153732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff2d2b-1cf3-42fc-a67f-b56661fef7b0 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:33.154093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff2d2b-1cf3-42fc-a67f-b56661fef7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:33.154365 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff2d2b-1cf3-42fc-a67f-b56661fef7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:33.177005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-01ff2d2b-1cf3-42fc-a67f-b56661fef7b0 None None] MFA Rules not processed for user `2e593646e79c443490b22657996b9108`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:33.178326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-01ff2d2b-1cf3-42fc-a67f-b56661fef7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:33.190689 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1093/2205] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:33.704378 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e06c3-ee12-4413-bd70-26593e360f47 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:33.705888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d91e06c3-ee12-4413-bd70-26593e360f47 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:33.779834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e06c3-ee12-4413-bd70-26593e360f47 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:33.786162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:33.786898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:33.791234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:33.791234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] PATH_INFO: `/v3/users/be4b02d5ae954712a5924bd9c2ad3555` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:33.791234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=be4b02d5ae954712a5924bd9c2ad3555)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:33.792979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:34.059746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be4b02d5ae954712a5924bd9c2ad3555'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:34.061195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be4b02d5ae954712a5924bd9c2ad3555'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:34.121513 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be4b02d5ae954712a5924bd9c2ad3555'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:34.122314 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be4b02d5ae954712a5924bd9c2ad3555'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:34.132188 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be4b02d5ae954712a5924bd9c2ad3555'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:34.162738 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1113/2206] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:33 2022] DELETE /identity/v3/users/be4b02d5ae954712a5924bd9c2ad3555 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:34.164169 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e000c51-22ff-40f1-962b-a59ac93cef47 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:34.164658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e000c51-22ff-40f1-962b-a59ac93cef47 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:34.164836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e000c51-22ff-40f1-962b-a59ac93cef47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:34.220888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-5e000c51-22ff-40f1-962b-a59ac93cef47 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:34.220888 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5e000c51-22ff-40f1-962b-a59ac93cef47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:34.316859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:34.353736 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1094/2207] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:34.362895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:34.365557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:34.454442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b081e57-c670-48b4-8b00-7769482b59ca admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:34.459483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7db4a6-99ad-4b81-86b7-c2809ac51317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:34.460332 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7db4a6-99ad-4b81-86b7-c2809ac51317 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:34.460960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7db4a6-99ad-4b81-86b7-c2809ac51317 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:34.461366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7db4a6-99ad-4b81-86b7-c2809ac51317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:34.475758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7db4a6-99ad-4b81-86b7-c2809ac51317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:34.477475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7db4a6-99ad-4b81-86b7-c2809ac51317 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:34.501871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1114/2208] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:35.159008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e000c51-22ff-40f1-962b-a59ac93cef47 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:35.161282 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5e000c51-22ff-40f1-962b-a59ac93cef47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:35.174871 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e000c51-22ff-40f1-962b-a59ac93cef47 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:35.181378 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:35.182153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:35.182661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:35.182941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] PATH_INFO: `/v3/projects/8325bc6ee4d3473e969b8c06de699f50` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:35.184224 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8325bc6ee4d3473e969b8c06de699f50)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:35.186207 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:35.344810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8325bc6ee4d3473e969b8c06de699f50', 'request_id': 'req-e06799ef-2def-4342-b6b4-e59b8cca7bc5'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:35.356425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8325bc6ee4d3473e969b8c06de699f50', 'request_id': 'req-e06799ef-2def-4342-b6b4-e59b8cca7bc5'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:35.371012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8325bc6ee4d3473e969b8c06de699f50', 'request_id': 'req-e06799ef-2def-4342-b6b4-e59b8cca7bc5'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:35.443543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] The token cache is being invalidate because project 8325bc6ee4d3473e969b8c06de699f50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:35.443543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:35.444540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:35.449750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1095/2209] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:35 2022] DELETE /identity/v3/projects/8325bc6ee4d3473e969b8c06de699f50 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:35.451465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7db4a6-99ad-4b81-86b7-c2809ac51317 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:35.464914 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7db4a6-99ad-4b81-86b7-c2809ac51317 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:35.480407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:35.571092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:35.571092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:35.571092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:35.571092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] PATH_INFO: `/v3/users/4a0c2525ff6b4127b3c3873baaf80452` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:35.571501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a0c2525ff6b4127b3c3873baaf80452)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:35.571501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:35.753409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a0c2525ff6b4127b3c3873baaf80452'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:35.754572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a0c2525ff6b4127b3c3873baaf80452'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:35.829602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a0c2525ff6b4127b3c3873baaf80452'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:35.830116 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a0c2525ff6b4127b3c3873baaf80452'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:35.840383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a0c2525ff6b4127b3c3873baaf80452'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:35.865472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1115/2210] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:35 2022] DELETE /identity/v3/users/4a0c2525ff6b4127b3c3873baaf80452 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:36.335445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:36.337016 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:36.389215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06799ef-2def-4342-b6b4-e59b8cca7bc5 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:36.502533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1139c02-f558-427c-976d-9ff1bd1d0666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:36.504691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1139c02-f558-427c-976d-9ff1bd1d0666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:36.505239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1139c02-f558-427c-976d-9ff1bd1d0666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:36.505800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1139c02-f558-427c-976d-9ff1bd1d0666 admin admin] PATH_INFO: `/v3/projects/88e85c6937b9477dbaea5664ea918bb4` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:36.507643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1139c02-f558-427c-976d-9ff1bd1d0666 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e85c6937b9477dbaea5664ea918bb4)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:36.510757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1139c02-f558-427c-976d-9ff1bd1d0666 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:36.648133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b1139c02-f558-427c-976d-9ff1bd1d0666 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e85c6937b9477dbaea5664ea918bb4', 'request_id': 'req-b1139c02-f558-427c-976d-9ff1bd1d0666'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:36.657443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b1139c02-f558-427c-976d-9ff1bd1d0666 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e85c6937b9477dbaea5664ea918bb4', 'request_id': 'req-b1139c02-f558-427c-976d-9ff1bd1d0666'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:36.665652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b1139c02-f558-427c-976d-9ff1bd1d0666 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e85c6937b9477dbaea5664ea918bb4', 'request_id': 'req-b1139c02-f558-427c-976d-9ff1bd1d0666'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:36.779317 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b1139c02-f558-427c-976d-9ff1bd1d0666 admin admin] The token cache is being invalidate because project 88e85c6937b9477dbaea5664ea918bb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:36.779978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b1139c02-f558-427c-976d-9ff1bd1d0666 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:36.781083 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b1139c02-f558-427c-976d-9ff1bd1d0666 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:36.783074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1096/2211] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:36 2022] DELETE /identity/v3/projects/88e85c6937b9477dbaea5664ea918bb4 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:36.787740 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:36.789176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:36.802427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c74242-881c-452c-83dc-85214bc25bc7 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:36.936863 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c2681-eadb-4c01-8514-f70d814a732b None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:36.937351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c2681-eadb-4c01-8514-f70d814a732b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:36.937767 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c2681-eadb-4c01-8514-f70d814a732b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:36.942990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ce4de-9daa-4996-8358-787a1de9d4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:36.942990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ce4de-9daa-4996-8358-787a1de9d4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:36.942990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ce4de-9daa-4996-8358-787a1de9d4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:36.942990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ce4de-9daa-4996-8358-787a1de9d4ca admin admin] PATH_INFO: `/v3/users/0e330a2ddf0547f9a0b930f4797a7d66` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:36.943384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ce4de-9daa-4996-8358-787a1de9d4ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e330a2ddf0547f9a0b930f4797a7d66)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:36.944467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ce4de-9daa-4996-8358-787a1de9d4ca admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:36.948168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:36.972891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-7f2c2681-eadb-4c01-8514-f70d814a732b None None] MFA Rules not processed for user `49fb50eb99b64f588c0312c2935c82fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:36.974269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7f2c2681-eadb-4c01-8514-f70d814a732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:37.057546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1097/2212] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:37.071543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c2681-eadb-4c01-8514-f70d814a732b None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:37.072941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7f2c2681-eadb-4c01-8514-f70d814a732b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:37.122384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:37.131881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-a50ce4de-9daa-4996-8358-787a1de9d4ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e330a2ddf0547f9a0b930f4797a7d66'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:37.132635 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-a50ce4de-9daa-4996-8358-787a1de9d4ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e330a2ddf0547f9a0b930f4797a7d66'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:37.152714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c2681-eadb-4c01-8514-f70d814a732b None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:37.158634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bf5e9-6581-4cf3-b752-e263ee6092f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:37.159397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bf5e9-6581-4cf3-b752-e263ee6092f5 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:37.159771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bf5e9-6581-4cf3-b752-e263ee6092f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:37.160230 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bf5e9-6581-4cf3-b752-e263ee6092f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:37.162389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-119bf5e9-6581-4cf3-b752-e263ee6092f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:37.200648 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-a50ce4de-9daa-4996-8358-787a1de9d4ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e330a2ddf0547f9a0b930f4797a7d66'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:37.201312 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-a50ce4de-9daa-4996-8358-787a1de9d4ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e330a2ddf0547f9a0b930f4797a7d66'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:37.221006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-a50ce4de-9daa-4996-8358-787a1de9d4ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e330a2ddf0547f9a0b930f4797a7d66'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:37.222487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bf5e9-6581-4cf3-b752-e263ee6092f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:37.224079 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119bf5e9-6581-4cf3-b752-e263ee6092f5 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:37.234013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1098/2213] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:44:37.243394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1116/2214] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:36 2022] DELETE /identity/v3/users/0e330a2ddf0547f9a0b930f4797a7d66 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:37.678728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bf5e9-6581-4cf3-b752-e263ee6092f5 service nova] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:37.678728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-119bf5e9-6581-4cf3-b752-e263ee6092f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:37.700286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bf5e9-6581-4cf3-b752-e263ee6092f5 service nova] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:37.800417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203884a5-80a2-4eed-aaf2-c7b7fee75872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:37.801093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203884a5-80a2-4eed-aaf2-c7b7fee75872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:37.801367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203884a5-80a2-4eed-aaf2-c7b7fee75872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:37.801678 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203884a5-80a2-4eed-aaf2-c7b7fee75872 admin admin] PATH_INFO: `/v3/projects/6fe5d9b0365e4bfa95b9e4f05d5fa57a` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:37.802899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203884a5-80a2-4eed-aaf2-c7b7fee75872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fe5d9b0365e4bfa95b9e4f05d5fa57a)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:37.804886 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203884a5-80a2-4eed-aaf2-c7b7fee75872 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:37.881635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:37.965473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-203884a5-80a2-4eed-aaf2-c7b7fee75872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fe5d9b0365e4bfa95b9e4f05d5fa57a', 'request_id': 'req-203884a5-80a2-4eed-aaf2-c7b7fee75872'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:37.971295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-203884a5-80a2-4eed-aaf2-c7b7fee75872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fe5d9b0365e4bfa95b9e4f05d5fa57a', 'request_id': 'req-203884a5-80a2-4eed-aaf2-c7b7fee75872'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:37.978735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-203884a5-80a2-4eed-aaf2-c7b7fee75872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fe5d9b0365e4bfa95b9e4f05d5fa57a', 'request_id': 'req-203884a5-80a2-4eed-aaf2-c7b7fee75872'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:38.035785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-203884a5-80a2-4eed-aaf2-c7b7fee75872 admin admin] The token cache is being invalidate because project 6fe5d9b0365e4bfa95b9e4f05d5fa57a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:38.036858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-203884a5-80a2-4eed-aaf2-c7b7fee75872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:38.037911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-203884a5-80a2-4eed-aaf2-c7b7fee75872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:38.040398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1099/2215] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:37 2022] DELETE /identity/v3/projects/6fe5d9b0365e4bfa95b9e4f05d5fa57a => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:38.045604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8530a9-3b19-4d53-86cd-fd4a5336f645 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:38.046096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8530a9-3b19-4d53-86cd-fd4a5336f645 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:38.046205 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8530a9-3b19-4d53-86cd-fd4a5336f645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:38.069877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-2f8530a9-3b19-4d53-86cd-fd4a5336f645 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:38.071101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2f8530a9-3b19-4d53-86cd-fd4a5336f645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:38.160795 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1117/2216] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:38.174267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1216e41-cd24-4ee1-8cc9-60a45b333e98 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:38.174559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1216e41-cd24-4ee1-8cc9-60a45b333e98 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:38.174823 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1216e41-cd24-4ee1-8cc9-60a45b333e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:38.196828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-d1216e41-cd24-4ee1-8cc9-60a45b333e98 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:38.196968 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d1216e41-cd24-4ee1-8cc9-60a45b333e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:38.205047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1100/2217] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:38.209477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8530a9-3b19-4d53-86cd-fd4a5336f645 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:38.219465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8530a9-3b19-4d53-86cd-fd4a5336f645 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:38.224746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242bd7b-030d-452c-a59c-358f229c539b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:38.227258 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242bd7b-030d-452c-a59c-358f229c539b admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:38.227258 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242bd7b-030d-452c-a59c-358f229c539b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:38.227258 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9242bd7b-030d-452c-a59c-358f229c539b admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:38.227258 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242bd7b-030d-452c-a59c-358f229c539b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:44:38.227756 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242bd7b-030d-452c-a59c-358f229c539b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:38.230737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9242bd7b-030d-452c-a59c-358f229c539b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:38.249889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1216e41-cd24-4ee1-8cc9-60a45b333e98 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:38.257942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1118/2218] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:44:38.262342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1216e41-cd24-4ee1-8cc9-60a45b333e98 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:38.278836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f02f0-66d8-4612-99a1-0f66a2078640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:38.278836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f02f0-66d8-4612-99a1-0f66a2078640 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:38.278836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f02f0-66d8-4612-99a1-0f66a2078640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:38.278836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f02f0-66d8-4612-99a1-0f66a2078640 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:38.279188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f02f0-66d8-4612-99a1-0f66a2078640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:38.279188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f02f0-66d8-4612-99a1-0f66a2078640 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:38.379706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1101/2219] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:38 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:38.383725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242bd7b-030d-452c-a59c-358f229c539b admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:38.393451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9242bd7b-030d-452c-a59c-358f229c539b admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:38.451071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49806b-1bfe-4ebf-b82f-746e62273323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:38.451405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49806b-1bfe-4ebf-b82f-746e62273323 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:38.451684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49806b-1bfe-4ebf-b82f-746e62273323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:38.451961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49806b-1bfe-4ebf-b82f-746e62273323 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:38.452605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49806b-1bfe-4ebf-b82f-746e62273323 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:38.454590 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49806b-1bfe-4ebf-b82f-746e62273323 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:38.560831 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1119/2220] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:38 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:38.565351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f02f0-66d8-4612-99a1-0f66a2078640 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:38.581583 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f02f0-66d8-4612-99a1-0f66a2078640 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:38.587471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1787ae9-6cce-4dfa-b086-8a220bf0ecfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:38.590114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1787ae9-6cce-4dfa-b086-8a220bf0ecfa admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:38.590114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1787ae9-6cce-4dfa-b086-8a220bf0ecfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:38.590114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1787ae9-6cce-4dfa-b086-8a220bf0ecfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:38.590114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1787ae9-6cce-4dfa-b086-8a220bf0ecfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:38.592352 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1787ae9-6cce-4dfa-b086-8a220bf0ecfa admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:38.610486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1102/2221] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:38.611706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49806b-1bfe-4ebf-b82f-746e62273323 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:38.622794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49806b-1bfe-4ebf-b82f-746e62273323 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:38.629852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb4809-e53e-4c5e-9e12-a2aba786542e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:38.631804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb4809-e53e-4c5e-9e12-a2aba786542e admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:38.631804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb4809-e53e-4c5e-9e12-a2aba786542e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:38.631804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb4809-e53e-4c5e-9e12-a2aba786542e admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:38.631804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb4809-e53e-4c5e-9e12-a2aba786542e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:38.633216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb4809-e53e-4c5e-9e12-a2aba786542e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:38.643874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1120/2222] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:44:38.648072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1787ae9-6cce-4dfa-b086-8a220bf0ecfa admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:38.658757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1787ae9-6cce-4dfa-b086-8a220bf0ecfa admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:38.663927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57c595-7f95-4b8a-93b1-6d373bb1b351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:38.664578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57c595-7f95-4b8a-93b1-6d373bb1b351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:38.664851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57c595-7f95-4b8a-93b1-6d373bb1b351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:38.665118 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad57c595-7f95-4b8a-93b1-6d373bb1b351 admin admin] PATH_INFO: `/v3/projects/d0146e2bcd4b4706b441530bec9970c4/users/758e407c2b054e25b2835015c24778b3/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:38.688454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57c595-7f95-4b8a-93b1-6d373bb1b351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0146e2bcd4b4706b441530bec9970c4, user_id=758e407c2b054e25b2835015c24778b3, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:38.691264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad57c595-7f95-4b8a-93b1-6d373bb1b351 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:38.791590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1103/2223] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:38 2022] PUT /identity/v3/projects/d0146e2bcd4b4706b441530bec9970c4/users/758e407c2b054e25b2835015c24778b3/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:44:38.802314 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8131759-22f2-4528-a53c-12b3f066daf3 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:38.802631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8131759-22f2-4528-a53c-12b3f066daf3 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:38.802847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8131759-22f2-4528-a53c-12b3f066daf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:38.846880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a8131759-22f2-4528-a53c-12b3f066daf3 None None] MFA Rules not processed for user `758e407c2b054e25b2835015c24778b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:38.849040 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a8131759-22f2-4528-a53c-12b3f066daf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:38.866443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:38.949974 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1121/2224] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:44:38.951592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57c595-7f95-4b8a-93b1-6d373bb1b351 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:39.110792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:39.146508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad57c595-7f95-4b8a-93b1-6d373bb1b351 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:39.165566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f11f2-48bf-4598-aa89-2034c1d06263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:39.165566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f11f2-48bf-4598-aa89-2034c1d06263 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:39.165566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f11f2-48bf-4598-aa89-2034c1d06263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:39.165566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f11f2-48bf-4598-aa89-2034c1d06263 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:39.167570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f11f2-48bf-4598-aa89-2034c1d06263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:39.171262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f11f2-48bf-4598-aa89-2034c1d06263 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:39.260860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1104/2225] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:38 2022] POST /identity/v3/projects => generated 385 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:39.266290 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8131759-22f2-4528-a53c-12b3f066daf3 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:39.281692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8131759-22f2-4528-a53c-12b3f066daf3 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:39.377927 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72965a64-202f-43ff-9e48-03b30c67ea58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:39.381112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72965a64-202f-43ff-9e48-03b30c67ea58 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:39.381112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72965a64-202f-43ff-9e48-03b30c67ea58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:39.381684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72965a64-202f-43ff-9e48-03b30c67ea58 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:39.382816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72965a64-202f-43ff-9e48-03b30c67ea58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:39.386465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72965a64-202f-43ff-9e48-03b30c67ea58 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:39.486347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1122/2226] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:39 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:39.488765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f11f2-48bf-4598-aa89-2034c1d06263 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:39.508892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f11f2-48bf-4598-aa89-2034c1d06263 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:39.538839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffee4e-6028-4e0f-860b-dc688855c691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:39.539618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffee4e-6028-4e0f-860b-dc688855c691 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:39.539995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffee4e-6028-4e0f-860b-dc688855c691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:39.540369 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffee4e-6028-4e0f-860b-dc688855c691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:39.541161 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ffee4e-6028-4e0f-860b-dc688855c691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:39.543800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ffee4e-6028-4e0f-860b-dc688855c691 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:39.568235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1105/2227] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:39.572599 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72965a64-202f-43ff-9e48-03b30c67ea58 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:39.584036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72965a64-202f-43ff-9e48-03b30c67ea58 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:39.589349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c073e-435a-44b4-9d6c-d23f4ca805cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:39.589798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c073e-435a-44b4-9d6c-d23f4ca805cc admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:39.590104 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c073e-435a-44b4-9d6c-d23f4ca805cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:39.590373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c073e-435a-44b4-9d6c-d23f4ca805cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:39.591013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c073e-435a-44b4-9d6c-d23f4ca805cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:39.592684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c073e-435a-44b4-9d6c-d23f4ca805cc admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:39.603927 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1123/2228] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:39.608217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffee4e-6028-4e0f-860b-dc688855c691 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:39.619247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffee4e-6028-4e0f-860b-dc688855c691 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:39.624270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139884d-433a-4e2b-aa48-93a6befa2944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:39.624936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139884d-433a-4e2b-aa48-93a6befa2944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:39.625219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139884d-433a-4e2b-aa48-93a6befa2944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:39.625898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139884d-433a-4e2b-aa48-93a6befa2944 admin admin] PATH_INFO: `/v3/projects/db59812a274942928f1f92efbc0caecd/users/beb261bdfef341e68ed7f731e4bca4a0/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:39.642422 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139884d-433a-4e2b-aa48-93a6befa2944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db59812a274942928f1f92efbc0caecd, user_id=beb261bdfef341e68ed7f731e4bca4a0, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:39.646426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139884d-433a-4e2b-aa48-93a6befa2944 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:39.742481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1106/2229] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:39 2022] PUT /identity/v3/projects/db59812a274942928f1f92efbc0caecd/users/beb261bdfef341e68ed7f731e4bca4a0/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:39.755886 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc04d86-bcdc-436d-b3c9-c34da3875de3 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:39.759741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc04d86-bcdc-436d-b3c9-c34da3875de3 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:39.759741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc04d86-bcdc-436d-b3c9-c34da3875de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:39.795964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-fcc04d86-bcdc-436d-b3c9-c34da3875de3 None None] MFA Rules not processed for user `beb261bdfef341e68ed7f731e4bca4a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:39.797276 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-fcc04d86-bcdc-436d-b3c9-c34da3875de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:39.880417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139884d-433a-4e2b-aa48-93a6befa2944 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:39.883196 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1139884d-433a-4e2b-aa48-93a6befa2944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:39.889539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1124/2230] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:39.898443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc04d86-bcdc-436d-b3c9-c34da3875de3 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:39.946266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:40.010461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139884d-433a-4e2b-aa48-93a6befa2944 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:40.016388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d991d6b-fa80-41ea-907b-7e3b18912a3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:40.017432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d991d6b-fa80-41ea-907b-7e3b18912a3b service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:40.018023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d991d6b-fa80-41ea-907b-7e3b18912a3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:40.018757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d991d6b-fa80-41ea-907b-7e3b18912a3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:40.022626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3d991d6b-fa80-41ea-907b-7e3b18912a3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:40.049487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc04d86-bcdc-436d-b3c9-c34da3875de3 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:40.065553 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28849c2f-0f69-40bf-b183-989fe28a2d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:40.068905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28849c2f-0f69-40bf-b183-989fe28a2d88 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:40.069255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28849c2f-0f69-40bf-b183-989fe28a2d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:40.069535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28849c2f-0f69-40bf-b183-989fe28a2d88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:40.076225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28849c2f-0f69-40bf-b183-989fe28a2d88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:40.090737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28849c2f-0f69-40bf-b183-989fe28a2d88 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:40.133787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d991d6b-fa80-41ea-907b-7e3b18912a3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:40.135456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d991d6b-fa80-41ea-907b-7e3b18912a3b service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:40.147395 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1107/2231] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:40.248606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1125/2232] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:39 2022] POST /identity/v3/projects => generated 385 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:40.252630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d991d6b-fa80-41ea-907b-7e3b18912a3b service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:40.274516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d991d6b-fa80-41ea-907b-7e3b18912a3b service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:40.337749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0097e8b-b0db-4773-b59e-03ffe39909af None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:40.338632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0097e8b-b0db-4773-b59e-03ffe39909af None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:40.339058 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0097e8b-b0db-4773-b59e-03ffe39909af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:40.366505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:40.376557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a0097e8b-b0db-4773-b59e-03ffe39909af None None] MFA Rules not processed for user `7c623b10040f458a92aa9db4d96d8271`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:40.378453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a0097e8b-b0db-4773-b59e-03ffe39909af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:40.385478 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70956f64-8442-444b-91d2-34cb2ea1e552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:40.386154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70956f64-8442-444b-91d2-34cb2ea1e552 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:40.386742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70956f64-8442-444b-91d2-34cb2ea1e552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:40.387195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70956f64-8442-444b-91d2-34cb2ea1e552 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:40.388555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70956f64-8442-444b-91d2-34cb2ea1e552 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:40.391263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70956f64-8442-444b-91d2-34cb2ea1e552 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:40.486293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1108/2233] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:40 2022] POST /identity/v3/users => generated 299 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:44:40.488190 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1126/2234] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:44:40.490665 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70956f64-8442-444b-91d2-34cb2ea1e552 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:40.506776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70956f64-8442-444b-91d2-34cb2ea1e552 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:40.532501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb9744-5748-479e-af83-9c9d28a4560a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:40.533356 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb9744-5748-479e-af83-9c9d28a4560a admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:40.533805 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb9744-5748-479e-af83-9c9d28a4560a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:40.534272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eb9744-5748-479e-af83-9c9d28a4560a admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:40.535668 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb9744-5748-479e-af83-9c9d28a4560a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:40.541184 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eb9744-5748-479e-af83-9c9d28a4560a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:40.556082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1109/2235] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:40.559998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0097e8b-b0db-4773-b59e-03ffe39909af None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:40.574519 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0097e8b-b0db-4773-b59e-03ffe39909af None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:40.579867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382b0c0-14de-4d1c-bb4a-1b5a01a4f68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:40.580647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382b0c0-14de-4d1c-bb4a-1b5a01a4f68a admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:40.582465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382b0c0-14de-4d1c-bb4a-1b5a01a4f68a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:40.582465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382b0c0-14de-4d1c-bb4a-1b5a01a4f68a admin admin] PATH_INFO: `/v3/projects/0d8bf8d6f1f34fc8a5b6d866cb32394f/users/1cd15051e9054130b0d35f83a7d7e8e3/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:40.607782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382b0c0-14de-4d1c-bb4a-1b5a01a4f68a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8bf8d6f1f34fc8a5b6d866cb32394f, user_id=1cd15051e9054130b0d35f83a7d7e8e3, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:40.610765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382b0c0-14de-4d1c-bb4a-1b5a01a4f68a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:40.693638 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1127/2236] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:40 2022] PUT /identity/v3/projects/0d8bf8d6f1f34fc8a5b6d866cb32394f/users/1cd15051e9054130b0d35f83a7d7e8e3/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:40.699398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb9744-5748-479e-af83-9c9d28a4560a admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:40.725302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eb9744-5748-479e-af83-9c9d28a4560a admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:40.825875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb549874-7b28-4918-b13d-6d39715b936a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:40.835130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb549874-7b28-4918-b13d-6d39715b936a admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:40.835130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb549874-7b28-4918-b13d-6d39715b936a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:40.835130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb549874-7b28-4918-b13d-6d39715b936a admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:40.835130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb549874-7b28-4918-b13d-6d39715b936a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:40.835130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb549874-7b28-4918-b13d-6d39715b936a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:40.866615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1110/2237] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:40 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:40.872284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382b0c0-14de-4d1c-bb4a-1b5a01a4f68a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:40.892610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382b0c0-14de-4d1c-bb4a-1b5a01a4f68a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:40.904743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177a293-7df0-4205-9bd8-f6fb38601a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:40.905575 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177a293-7df0-4205-9bd8-f6fb38601a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:40.906166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177a293-7df0-4205-9bd8-f6fb38601a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:40.907451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177a293-7df0-4205-9bd8-f6fb38601a23 admin admin] PATH_INFO: `/v3/projects/0d8bf8d6f1f34fc8a5b6d866cb32394f/users/1cd15051e9054130b0d35f83a7d7e8e3/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:40.918011 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177a293-7df0-4205-9bd8-f6fb38601a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d8bf8d6f1f34fc8a5b6d866cb32394f, user_id=1cd15051e9054130b0d35f83a7d7e8e3, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:40.925084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177a293-7df0-4205-9bd8-f6fb38601a23 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:40.939202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1128/2238] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:40 2022] PUT /identity/v3/projects/0d8bf8d6f1f34fc8a5b6d866cb32394f/users/1cd15051e9054130b0d35f83a7d7e8e3/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:40.953215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda75614-4e82-49a7-abe8-1b914ee4c730 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:40.953570 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda75614-4e82-49a7-abe8-1b914ee4c730 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:40.955641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda75614-4e82-49a7-abe8-1b914ee4c730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:41.023824 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-fda75614-4e82-49a7-abe8-1b914ee4c730 None None] MFA Rules not processed for user `1cd15051e9054130b0d35f83a7d7e8e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:41.024992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-fda75614-4e82-49a7-abe8-1b914ee4c730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:41.154175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1111/2239] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:41.617051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177a293-7df0-4205-9bd8-f6fb38601a23 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:41.722006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177a293-7df0-4205-9bd8-f6fb38601a23 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:41.729695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc17b16-2b82-4f20-9296-dc8bac6efb9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:41.730688 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc17b16-2b82-4f20-9296-dc8bac6efb9b service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:41.731162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc17b16-2b82-4f20-9296-dc8bac6efb9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:41.731434 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc17b16-2b82-4f20-9296-dc8bac6efb9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:41.762791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:41.836132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc17b16-2b82-4f20-9296-dc8bac6efb9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:41.853704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc17b16-2b82-4f20-9296-dc8bac6efb9b service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:41.902181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1129/2240] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:46.203666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda75614-4e82-49a7-abe8-1b914ee4c730 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:46.214590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda75614-4e82-49a7-abe8-1b914ee4c730 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:46.217557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22939f-0c43-4f85-b8ec-8c283f83bcc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:46.218190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22939f-0c43-4f85-b8ec-8c283f83bcc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:46.218463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22939f-0c43-4f85-b8ec-8c283f83bcc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:46.218727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22939f-0c43-4f85-b8ec-8c283f83bcc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:46.277810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:46.280536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22939f-0c43-4f85-b8ec-8c283f83bcc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:46.282008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22939f-0c43-4f85-b8ec-8c283f83bcc2 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:46.292567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1112/2241] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:47.821878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc17b16-2b82-4f20-9296-dc8bac6efb9b service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:47.830585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc17b16-2b82-4f20-9296-dc8bac6efb9b service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:47.834219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e17394-5fb8-41f3-ab22-1d6dbe4b99d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:47.835033 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e17394-5fb8-41f3-ab22-1d6dbe4b99d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:47.835314 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e17394-5fb8-41f3-ab22-1d6dbe4b99d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:47.835577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e17394-5fb8-41f3-ab22-1d6dbe4b99d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:47.845178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e17394-5fb8-41f3-ab22-1d6dbe4b99d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:47.846614 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e17394-5fb8-41f3-ab22-1d6dbe4b99d4 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:47.858823 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1130/2242] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:47 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:49.653867 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e45d1c-d5b8-4404-aa53-38c2ac92db96 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:49.654372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e45d1c-d5b8-4404-aa53-38c2ac92db96 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:49.654562 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e45d1c-d5b8-4404-aa53-38c2ac92db96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:49.674323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-d4e45d1c-d5b8-4404-aa53-38c2ac92db96 None None] MFA Rules not processed for user `12e7e80742cc426da9984af75016a890`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:49.675557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d4e45d1c-d5b8-4404-aa53-38c2ac92db96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:49.741544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1113/2243] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:54.994589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e17394-5fb8-41f3-ab22-1d6dbe4b99d4 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:55.087949 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e17394-5fb8-41f3-ab22-1d6dbe4b99d4 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:55.102749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:55.107232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:55.107232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:55.107232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] PATH_INFO: `/v3/users/758e407c2b054e25b2835015c24778b3` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:55.107232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=758e407c2b054e25b2835015c24778b3)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:55.107921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:55.243435 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '758e407c2b054e25b2835015c24778b3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:55.244014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '758e407c2b054e25b2835015c24778b3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:55.291773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '758e407c2b054e25b2835015c24778b3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:55.292383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '758e407c2b054e25b2835015c24778b3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:55.299582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '758e407c2b054e25b2835015c24778b3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:55.317732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1131/2244] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:54 2022] DELETE /identity/v3/users/758e407c2b054e25b2835015c24778b3 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:55.322564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6361415-ae3d-4e6b-bbc1-09662d1e9839 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:55.323030 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6361415-ae3d-4e6b-bbc1-09662d1e9839 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:55.323145 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6361415-ae3d-4e6b-bbc1-09662d1e9839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:55.382044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c6361415-ae3d-4e6b-bbc1-09662d1e9839 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:55.390020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c6361415-ae3d-4e6b-bbc1-09662d1e9839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:55.465248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1114/2245] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:55.473488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:55.475184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:55.528707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:55.554557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed44556-b8c0-450c-b5f1-90015dfc25e8 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:55.558371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8262e-64c1-48b4-ab87-344499ea1925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:55.559063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8262e-64c1-48b4-ab87-344499ea1925 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:55.559402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8262e-64c1-48b4-ab87-344499ea1925 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:55.559758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8262e-64c1-48b4-ab87-344499ea1925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:55.571555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8262e-64c1-48b4-ab87-344499ea1925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:55.573019 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8262e-64c1-48b4-ab87-344499ea1925 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:55.584091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1132/2246] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:44:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:44:56.110337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6361415-ae3d-4e6b-bbc1-09662d1e9839 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:56.112383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c6361415-ae3d-4e6b-bbc1-09662d1e9839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:56.127331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6361415-ae3d-4e6b-bbc1-09662d1e9839 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:56.133324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:56.134293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:56.134293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:56.134293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] PATH_INFO: `/v3/projects/d0146e2bcd4b4706b441530bec9970c4` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:56.137357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0146e2bcd4b4706b441530bec9970c4)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:56.139285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:56.249652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0146e2bcd4b4706b441530bec9970c4', 'request_id': 'req-aacca888-8759-438c-b9c4-fc59f14fe0be'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:56.258674 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0146e2bcd4b4706b441530bec9970c4', 'request_id': 'req-aacca888-8759-438c-b9c4-fc59f14fe0be'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:56.265612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0146e2bcd4b4706b441530bec9970c4', 'request_id': 'req-aacca888-8759-438c-b9c4-fc59f14fe0be'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:56.354748 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] The token cache is being invalidate because project d0146e2bcd4b4706b441530bec9970c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:56.355443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:56.355723 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:56.357351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1115/2247] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:56 2022] DELETE /identity/v3/projects/d0146e2bcd4b4706b441530bec9970c4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:56.361436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8262e-64c1-48b4-ab87-344499ea1925 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:56.371677 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8262e-64c1-48b4-ab87-344499ea1925 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:56.438372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:56.443680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:56.443680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:56.443680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] PATH_INFO: `/v3/users/beb261bdfef341e68ed7f731e4bca4a0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:56.443680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beb261bdfef341e68ed7f731e4bca4a0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:56.443680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:56.614609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beb261bdfef341e68ed7f731e4bca4a0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:56.615254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beb261bdfef341e68ed7f731e4bca4a0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:56.668993 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beb261bdfef341e68ed7f731e4bca4a0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:56.669641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beb261bdfef341e68ed7f731e4bca4a0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:56.678453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beb261bdfef341e68ed7f731e4bca4a0'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:56.707608 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1133/2248] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:56 2022] DELETE /identity/v3/users/beb261bdfef341e68ed7f731e4bca4a0 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:57.185227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:57.186739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:57.200673 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacca888-8759-438c-b9c4-fc59f14fe0be admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:57.224758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:57.264649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:57.265269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:57.265540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:57.265802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] PATH_INFO: `/v3/projects/db59812a274942928f1f92efbc0caecd` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:57.266920 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db59812a274942928f1f92efbc0caecd)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:57.268793 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:57.378751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db59812a274942928f1f92efbc0caecd', 'request_id': 'req-18d6a46c-f40c-4341-b909-01556f92bd01'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:57.384652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db59812a274942928f1f92efbc0caecd', 'request_id': 'req-18d6a46c-f40c-4341-b909-01556f92bd01'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:57.390878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db59812a274942928f1f92efbc0caecd', 'request_id': 'req-18d6a46c-f40c-4341-b909-01556f92bd01'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:57.463768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] The token cache is being invalidate because project db59812a274942928f1f92efbc0caecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:57.464057 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:57.464788 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:57.466625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1116/2249] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:57 2022] DELETE /identity/v3/projects/db59812a274942928f1f92efbc0caecd => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:57.470856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:57.472713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:57.486067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415baf8b-a881-497a-ad1b-c4b929711b98 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:57.535172 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:57.544472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b8745-ef42-4ebb-b8bb-1a1fc5e5d1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:57.545101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b8745-ef42-4ebb-b8bb-1a1fc5e5d1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:57.545379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b8745-ef42-4ebb-b8bb-1a1fc5e5d1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:57.545650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b8745-ef42-4ebb-b8bb-1a1fc5e5d1aa admin admin] PATH_INFO: `/v3/users/1cd15051e9054130b0d35f83a7d7e8e3` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:57.546800 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b8745-ef42-4ebb-b8bb-1a1fc5e5d1aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd15051e9054130b0d35f83a7d7e8e3)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:57.548711 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b8745-ef42-4ebb-b8bb-1a1fc5e5d1aa admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:57.682487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4e4b8745-ef42-4ebb-b8bb-1a1fc5e5d1aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd15051e9054130b0d35f83a7d7e8e3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:57.683061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4e4b8745-ef42-4ebb-b8bb-1a1fc5e5d1aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd15051e9054130b0d35f83a7d7e8e3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:57.737275 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4e4b8745-ef42-4ebb-b8bb-1a1fc5e5d1aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd15051e9054130b0d35f83a7d7e8e3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:57.737924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4e4b8745-ef42-4ebb-b8bb-1a1fc5e5d1aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd15051e9054130b0d35f83a7d7e8e3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:57.745810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-4e4b8745-ef42-4ebb-b8bb-1a1fc5e5d1aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd15051e9054130b0d35f83a7d7e8e3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:57.796571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1134/2250] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:44:57 2022] DELETE /identity/v3/users/1cd15051e9054130b0d35f83a7d7e8e3 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:58.385779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:58.387563 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:58.400546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6a46c-f40c-4341-b909-01556f92bd01 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:58.500066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c1a33-654a-4087-be23-a5918ab65466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:58.500702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c1a33-654a-4087-be23-a5918ab65466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:58.501234 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c1a33-654a-4087-be23-a5918ab65466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:58.501234 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c1a33-654a-4087-be23-a5918ab65466 admin admin] PATH_INFO: `/v3/projects/0d8bf8d6f1f34fc8a5b6d866cb32394f` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:58.502994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c1a33-654a-4087-be23-a5918ab65466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d8bf8d6f1f34fc8a5b6d866cb32394f)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:58.505683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c1a33-654a-4087-be23-a5918ab65466 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:58.625265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-dc9c1a33-654a-4087-be23-a5918ab65466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8bf8d6f1f34fc8a5b6d866cb32394f', 'request_id': 'req-dc9c1a33-654a-4087-be23-a5918ab65466'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:58.628061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:58.632874 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-dc9c1a33-654a-4087-be23-a5918ab65466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8bf8d6f1f34fc8a5b6d866cb32394f', 'request_id': 'req-dc9c1a33-654a-4087-be23-a5918ab65466'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:58.639553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-dc9c1a33-654a-4087-be23-a5918ab65466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8bf8d6f1f34fc8a5b6d866cb32394f', 'request_id': 'req-dc9c1a33-654a-4087-be23-a5918ab65466'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:58.691053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-dc9c1a33-654a-4087-be23-a5918ab65466 admin admin] The token cache is being invalidate because project 0d8bf8d6f1f34fc8a5b6d866cb32394f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:44:58.691335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-dc9c1a33-654a-4087-be23-a5918ab65466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:58.692445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-dc9c1a33-654a-4087-be23-a5918ab65466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:44:58.693979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1117/2251] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:44:58 2022] DELETE /identity/v3/projects/0d8bf8d6f1f34fc8a5b6d866cb32394f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:58.699713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fec58-77ed-4d68-a6d0-4094f62ebf69 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:58.700202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fec58-77ed-4d68-a6d0-4094f62ebf69 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:58.700293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13fec58-77ed-4d68-a6d0-4094f62ebf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:58.721987 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a13fec58-77ed-4d68-a6d0-4094f62ebf69 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:58.723228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a13fec58-77ed-4d68-a6d0-4094f62ebf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:58.799311 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1135/2252] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:58.808415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b34ab9-3cd2-48e5-9bd6-dd3fb396054c None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:58.808700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b34ab9-3cd2-48e5-9bd6-dd3fb396054c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:58.809256 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b34ab9-3cd2-48e5-9bd6-dd3fb396054c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:58.831433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-a1b34ab9-3cd2-48e5-9bd6-dd3fb396054c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:58.833812 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a1b34ab9-3cd2-48e5-9bd6-dd3fb396054c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:58.849263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1118/2253] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:44:58.850770 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fec58-77ed-4d68-a6d0-4094f62ebf69 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:58.866271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13fec58-77ed-4d68-a6d0-4094f62ebf69 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:58.873654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48682bb6-1463-4faf-943b-69ff9ee2e652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:58.874288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48682bb6-1463-4faf-943b-69ff9ee2e652 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:58.874567 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48682bb6-1463-4faf-943b-69ff9ee2e652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:58.874835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48682bb6-1463-4faf-943b-69ff9ee2e652 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:58.875485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48682bb6-1463-4faf-943b-69ff9ee2e652 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:44:58.875786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48682bb6-1463-4faf-943b-69ff9ee2e652 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:58.877542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48682bb6-1463-4faf-943b-69ff9ee2e652 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:58.893259 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1136/2254] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:44:58.895972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b34ab9-3cd2-48e5-9bd6-dd3fb396054c None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:58.905923 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b34ab9-3cd2-48e5-9bd6-dd3fb396054c None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:58.910823 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5f218-a1ee-4d45-98ed-71bd61dd6115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:58.911432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5f218-a1ee-4d45-98ed-71bd61dd6115 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:58.911708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5f218-a1ee-4d45-98ed-71bd61dd6115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:58.911969 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d5f218-a1ee-4d45-98ed-71bd61dd6115 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:58.912604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5f218-a1ee-4d45-98ed-71bd61dd6115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:58.914524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d5f218-a1ee-4d45-98ed-71bd61dd6115 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:59.264599 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1119/2255] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:44:58 2022] POST /identity/v3/projects => generated 369 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:44:59.264818 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48682bb6-1463-4faf-943b-69ff9ee2e652 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:59.274204 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48682bb6-1463-4faf-943b-69ff9ee2e652 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:59.362893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b618a-d673-4df5-9869-0bd55945ad3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:59.364215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b618a-d673-4df5-9869-0bd55945ad3e admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:59.367199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b618a-d673-4df5-9869-0bd55945ad3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:59.367710 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b618a-d673-4df5-9869-0bd55945ad3e admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:59.368588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b618a-d673-4df5-9869-0bd55945ad3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:59.370723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b618a-d673-4df5-9869-0bd55945ad3e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:59.382958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:44:59.458492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1137/2256] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:44:59 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:44:59.462875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5f218-a1ee-4d45-98ed-71bd61dd6115 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:59.474542 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d5f218-a1ee-4d45-98ed-71bd61dd6115 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:59.479365 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501fee7-0cef-4c33-8a23-3b75b7dabbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:59.480029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501fee7-0cef-4c33-8a23-3b75b7dabbbb admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:59.480306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501fee7-0cef-4c33-8a23-3b75b7dabbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:59.480573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e501fee7-0cef-4c33-8a23-3b75b7dabbbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:59.481211 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501fee7-0cef-4c33-8a23-3b75b7dabbbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:59.482845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e501fee7-0cef-4c33-8a23-3b75b7dabbbb admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:59.497098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1120/2257] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:59.501478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b618a-d673-4df5-9869-0bd55945ad3e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:59.514197 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b618a-d673-4df5-9869-0bd55945ad3e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:59.519062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06581b5a-4364-432f-9fa4-3c0145f25d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:59.519752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06581b5a-4364-432f-9fa4-3c0145f25d7c admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:59.520092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06581b5a-4364-432f-9fa4-3c0145f25d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:59.520868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06581b5a-4364-432f-9fa4-3c0145f25d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:59.521581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06581b5a-4364-432f-9fa4-3c0145f25d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:59.523408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06581b5a-4364-432f-9fa4-3c0145f25d7c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:59.537738 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1138/2258] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:44:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:44:59.541398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501fee7-0cef-4c33-8a23-3b75b7dabbbb admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:44:59.553195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e501fee7-0cef-4c33-8a23-3b75b7dabbbb admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:44:59.565264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6517f1-162d-4de3-b2a4-a880aedb7685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:44:59.566111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6517f1-162d-4de3-b2a4-a880aedb7685 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:59.566386 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6517f1-162d-4de3-b2a4-a880aedb7685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:59.567008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6517f1-162d-4de3-b2a4-a880aedb7685 admin admin] PATH_INFO: `/v3/projects/ce5edc1d496a4318aba96b0df7b3ba68/users/85db699157c1461e92cc42fe3a86a150/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:59.586909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6517f1-162d-4de3-b2a4-a880aedb7685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce5edc1d496a4318aba96b0df7b3ba68, user_id=85db699157c1461e92cc42fe3a86a150, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:44:59.589601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6517f1-162d-4de3-b2a4-a880aedb7685 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:44:59.665598 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1121/2259] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:44:59 2022] PUT /identity/v3/projects/ce5edc1d496a4318aba96b0df7b3ba68/users/85db699157c1461e92cc42fe3a86a150/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:44:59.676131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482fd5e-0d0e-491b-8600-f13d6671fcb5 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:44:59.676866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482fd5e-0d0e-491b-8600-f13d6671fcb5 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:44:59.677149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482fd5e-0d0e-491b-8600-f13d6671fcb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:44:59.731835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-8482fd5e-0d0e-491b-8600-f13d6671fcb5 None None] MFA Rules not processed for user `85db699157c1461e92cc42fe3a86a150`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:44:59.733349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8482fd5e-0d0e-491b-8600-f13d6671fcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:44:59.842441 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1139/2260] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:44:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:07.909357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6517f1-162d-4de3-b2a4-a880aedb7685 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:07.911246 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-ec6517f1-162d-4de3-b2a4-a880aedb7685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:07.974508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6517f1-162d-4de3-b2a4-a880aedb7685 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:07.978152 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26925ac9-137e-482b-92b5-535054fbf604 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d96ec6235843c5b67d87a3d0ff0f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:07.978813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26925ac9-137e-482b-92b5-535054fbf604 service glance] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:07.979011 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26925ac9-137e-482b-92b5-535054fbf604 service glance] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:07.979278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26925ac9-137e-482b-92b5-535054fbf604 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:07.987761 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26925ac9-137e-482b-92b5-535054fbf604 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:07.989176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26925ac9-137e-482b-92b5-535054fbf604 service glance] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:07.999485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1122/2261] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:09.832409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:12.975882 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482fd5e-0d0e-491b-8600-f13d6671fcb5 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:12.977323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8482fd5e-0d0e-491b-8600-f13d6671fcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:13.056155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482fd5e-0d0e-491b-8600-f13d6671fcb5 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:13.060892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:13.062713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:13.062713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:13.062713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] PATH_INFO: `/v3/users/327a7f16283a49d2b4eefab257baa502` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:13.063450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=327a7f16283a49d2b4eefab257baa502)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:13.065451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:13.224516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '327a7f16283a49d2b4eefab257baa502'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:13.225359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '327a7f16283a49d2b4eefab257baa502'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:13.285749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '327a7f16283a49d2b4eefab257baa502'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:13.286538 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '327a7f16283a49d2b4eefab257baa502'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:13.295065 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '327a7f16283a49d2b4eefab257baa502'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:13.317714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1140/2262] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:45:12 2022] DELETE /identity/v3/users/327a7f16283a49d2b4eefab257baa502 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:13.317936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf57581-77b5-463f-b219-979c15e42cf9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:13.317936 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf57581-77b5-463f-b219-979c15e42cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:13.319043 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf57581-77b5-463f-b219-979c15e42cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:13.338265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-7cf57581-77b5-463f-b219-979c15e42cf9 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:13.339492 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7cf57581-77b5-463f-b219-979c15e42cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:13.402358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1123/2263] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:13.412446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:13.414169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:13.503434 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643d58d-02b2-4dd0-9b75-e24934bbdfd1 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:13.508887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bc718-be6e-4941-892f-510e8a8e66e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:13.509890 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bc718-be6e-4941-892f-510e8a8e66e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:13.510376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bc718-be6e-4941-892f-510e8a8e66e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:13.510645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bc718-be6e-4941-892f-510e8a8e66e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:13.521597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165bc718-be6e-4941-892f-510e8a8e66e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:13.523057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165bc718-be6e-4941-892f-510e8a8e66e7 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:13.535870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1141/2264] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:13.904476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf57581-77b5-463f-b219-979c15e42cf9 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:13.911712 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7cf57581-77b5-463f-b219-979c15e42cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:13.922139 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf57581-77b5-463f-b219-979c15e42cf9 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:13.927080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:13.928211 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:13.928487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:13.928757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] PATH_INFO: `/v3/projects/577338b7d2b44b3e9ba41d6b1120dd8e` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:13.934166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=577338b7d2b44b3e9ba41d6b1120dd8e)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:13.934166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:13.948944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:14.046651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '577338b7d2b44b3e9ba41d6b1120dd8e', 'request_id': 'req-941d64a5-2775-452b-a607-17bf56b669ac'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:14.053413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '577338b7d2b44b3e9ba41d6b1120dd8e', 'request_id': 'req-941d64a5-2775-452b-a607-17bf56b669ac'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:14.059720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '577338b7d2b44b3e9ba41d6b1120dd8e', 'request_id': 'req-941d64a5-2775-452b-a607-17bf56b669ac'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:14.141968 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] The token cache is being invalidate because project 577338b7d2b44b3e9ba41d6b1120dd8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:45:14.142275 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:14.142955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:14.144453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1124/2265] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:45:13 2022] DELETE /identity/v3/projects/577338b7d2b44b3e9ba41d6b1120dd8e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:14.149613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bc718-be6e-4941-892f-510e8a8e66e7 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:14.160204 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bc718-be6e-4941-892f-510e8a8e66e7 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:14.181244 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:14.227409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a410bdf-ac7c-445c-8c42-7a78f0d142de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:14.228617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a410bdf-ac7c-445c-8c42-7a78f0d142de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:14.229156 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a410bdf-ac7c-445c-8c42-7a78f0d142de admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:14.229565 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a410bdf-ac7c-445c-8c42-7a78f0d142de admin admin] PATH_INFO: `/v3/users/df3c3f11c4c04471a78ee5518976fbcc` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:14.231069 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a410bdf-ac7c-445c-8c42-7a78f0d142de admin admin] RBAC: Authorizing `identity:delete_user(user_id=df3c3f11c4c04471a78ee5518976fbcc)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:14.233548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a410bdf-ac7c-445c-8c42-7a78f0d142de admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:14.445464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9a410bdf-ac7c-445c-8c42-7a78f0d142de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df3c3f11c4c04471a78ee5518976fbcc'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:14.446734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9a410bdf-ac7c-445c-8c42-7a78f0d142de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df3c3f11c4c04471a78ee5518976fbcc'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:14.531200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9a410bdf-ac7c-445c-8c42-7a78f0d142de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df3c3f11c4c04471a78ee5518976fbcc'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:14.531828 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9a410bdf-ac7c-445c-8c42-7a78f0d142de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df3c3f11c4c04471a78ee5518976fbcc'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:14.541696 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9a410bdf-ac7c-445c-8c42-7a78f0d142de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df3c3f11c4c04471a78ee5518976fbcc'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:14.560187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1142/2266] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:45:14 2022] DELETE /identity/v3/users/df3c3f11c4c04471a78ee5518976fbcc => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:15.002809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:15.004386 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:15.022492 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d64a5-2775-452b-a607-17bf56b669ac admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:15.092892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0638d9f-b0b6-447e-a550-fdce3e553427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:15.093530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0638d9f-b0b6-447e-a550-fdce3e553427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:15.093801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0638d9f-b0b6-447e-a550-fdce3e553427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:15.094101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0638d9f-b0b6-447e-a550-fdce3e553427 admin admin] PATH_INFO: `/v3/projects/deef889de876421b96f65d14d05f6344` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:15.096010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0638d9f-b0b6-447e-a550-fdce3e553427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deef889de876421b96f65d14d05f6344)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:15.097944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0638d9f-b0b6-447e-a550-fdce3e553427 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:15.215600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c0638d9f-b0b6-447e-a550-fdce3e553427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deef889de876421b96f65d14d05f6344', 'request_id': 'req-c0638d9f-b0b6-447e-a550-fdce3e553427'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:15.221457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c0638d9f-b0b6-447e-a550-fdce3e553427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deef889de876421b96f65d14d05f6344', 'request_id': 'req-c0638d9f-b0b6-447e-a550-fdce3e553427'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:15.230611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c0638d9f-b0b6-447e-a550-fdce3e553427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deef889de876421b96f65d14d05f6344', 'request_id': 'req-c0638d9f-b0b6-447e-a550-fdce3e553427'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:15.334957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c0638d9f-b0b6-447e-a550-fdce3e553427 admin admin] The token cache is being invalidate because project deef889de876421b96f65d14d05f6344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:45:15.335340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c0638d9f-b0b6-447e-a550-fdce3e553427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:15.336439 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c0638d9f-b0b6-447e-a550-fdce3e553427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:15.339105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1125/2267] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:45:15 2022] DELETE /identity/v3/projects/deef889de876421b96f65d14d05f6344 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:15.351878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269309a5-bd1c-4014-a957-750fb46b7da3 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:15.352414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269309a5-bd1c-4014-a957-750fb46b7da3 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:15.356858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269309a5-bd1c-4014-a957-750fb46b7da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:15.377645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-269309a5-bd1c-4014-a957-750fb46b7da3 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:15.377645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-269309a5-bd1c-4014-a957-750fb46b7da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:15.457743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1143/2268] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:45:15.470475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53e8a9-0ed6-47d0-a7df-7358f7091dbe None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:15.470751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53e8a9-0ed6-47d0-a7df-7358f7091dbe None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:15.475634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53e8a9-0ed6-47d0-a7df-7358f7091dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:15.530694 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-7e53e8a9-0ed6-47d0-a7df-7358f7091dbe None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:15.538057 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7e53e8a9-0ed6-47d0-a7df-7358f7091dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:15.562839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1126/2269] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:45:15.568800 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269309a5-bd1c-4014-a957-750fb46b7da3 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:15.585692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269309a5-bd1c-4014-a957-750fb46b7da3 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:15.602914 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1366047-fcb9-4693-b756-4b57e88844b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:15.603716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1366047-fcb9-4693-b756-4b57e88844b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:15.604538 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1366047-fcb9-4693-b756-4b57e88844b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:15.605141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1366047-fcb9-4693-b756-4b57e88844b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:15.606815 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1366047-fcb9-4693-b756-4b57e88844b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:45:15.608717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1366047-fcb9-4693-b756-4b57e88844b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:15.610687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1366047-fcb9-4693-b756-4b57e88844b1 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:15.629543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1144/2270] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:45:15.637782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53e8a9-0ed6-47d0-a7df-7358f7091dbe None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:15.658410 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53e8a9-0ed6-47d0-a7df-7358f7091dbe None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:15.673580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f0f12-0af0-433a-9001-3811bf2fa5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:15.674249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f0f12-0af0-433a-9001-3811bf2fa5e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:15.674521 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f0f12-0af0-433a-9001-3811bf2fa5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:15.674785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f0f12-0af0-433a-9001-3811bf2fa5e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:15.675426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f0f12-0af0-433a-9001-3811bf2fa5e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:15.677340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f0f12-0af0-433a-9001-3811bf2fa5e9 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:15.768765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1127/2271] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:45:15 2022] POST /identity/v3/projects => generated 399 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:45:15.774124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1366047-fcb9-4693-b756-4b57e88844b1 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:15.785563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1366047-fcb9-4693-b756-4b57e88844b1 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:15.819529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:15.869222 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a3aae-09f4-4bd4-b42c-1ed1e72fc2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:15.869932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a3aae-09f4-4bd4-b42c-1ed1e72fc2a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:15.871183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a3aae-09f4-4bd4-b42c-1ed1e72fc2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:15.871183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a3aae-09f4-4bd4-b42c-1ed1e72fc2a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:15.871823 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a3aae-09f4-4bd4-b42c-1ed1e72fc2a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:15.874972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a3aae-09f4-4bd4-b42c-1ed1e72fc2a7 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:15.996503 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1145/2272] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:45:15 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:15.998079 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f0f12-0af0-433a-9001-3811bf2fa5e9 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:16.005331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:16.010585 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f0f12-0af0-433a-9001-3811bf2fa5e9 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:16.015580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b1e8f-8938-409c-ba63-e46860d9849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:16.016238 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b1e8f-8938-409c-ba63-e46860d9849e admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:16.016512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b1e8f-8938-409c-ba63-e46860d9849e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:16.016782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b1e8f-8938-409c-ba63-e46860d9849e admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:16.017463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b1e8f-8938-409c-ba63-e46860d9849e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:16.019096 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b1e8f-8938-409c-ba63-e46860d9849e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:16.031279 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1128/2273] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:16.035230 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a3aae-09f4-4bd4-b42c-1ed1e72fc2a7 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:16.046446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a3aae-09f4-4bd4-b42c-1ed1e72fc2a7 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:16.051478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d0ac7-65bf-426d-8616-a2e978fcc92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:16.052087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d0ac7-65bf-426d-8616-a2e978fcc92a admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:16.052451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d0ac7-65bf-426d-8616-a2e978fcc92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:16.052719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d0ac7-65bf-426d-8616-a2e978fcc92a admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:16.053362 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d0ac7-65bf-426d-8616-a2e978fcc92a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:16.055001 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d0ac7-65bf-426d-8616-a2e978fcc92a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:16.071991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1146/2274] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:16.073294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b1e8f-8938-409c-ba63-e46860d9849e admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:16.085401 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b1e8f-8938-409c-ba63-e46860d9849e admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:16.090232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4069ba-31c6-4dac-94dc-fa0b99ffee7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:16.102455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4069ba-31c6-4dac-94dc-fa0b99ffee7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:16.102455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4069ba-31c6-4dac-94dc-fa0b99ffee7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:16.102455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4069ba-31c6-4dac-94dc-fa0b99ffee7c admin admin] PATH_INFO: `/v3/projects/bce073fbd9234c909f533c059fc949fe/users/483ec8865ec3471e9ac6867e91ff99f5/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:16.108158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4069ba-31c6-4dac-94dc-fa0b99ffee7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce073fbd9234c909f533c059fc949fe, user_id=483ec8865ec3471e9ac6867e91ff99f5, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:16.110850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4069ba-31c6-4dac-94dc-fa0b99ffee7c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:16.213550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1129/2275] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:45:16 2022] PUT /identity/v3/projects/bce073fbd9234c909f533c059fc949fe/users/483ec8865ec3471e9ac6867e91ff99f5/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:16.223582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94576aca-52dd-48ae-97d5-870d84c901cc None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:16.223859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94576aca-52dd-48ae-97d5-870d84c901cc None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:16.224273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94576aca-52dd-48ae-97d5-870d84c901cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:16.255149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-94576aca-52dd-48ae-97d5-870d84c901cc None None] MFA Rules not processed for user `483ec8865ec3471e9ac6867e91ff99f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:16.257576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-94576aca-52dd-48ae-97d5-870d84c901cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:16.345526 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1147/2276] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:16 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:45:16.351112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4069ba-31c6-4dac-94dc-fa0b99ffee7c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:16.432992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4069ba-31c6-4dac-94dc-fa0b99ffee7c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:16.441325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233d975-3cf9-406d-ade5-a7afadf10afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:16.441985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233d975-3cf9-406d-ade5-a7afadf10afe admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:16.442260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233d975-3cf9-406d-ade5-a7afadf10afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:16.442524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233d975-3cf9-406d-ade5-a7afadf10afe admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:16.443367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233d975-3cf9-406d-ade5-a7afadf10afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:16.445261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233d975-3cf9-406d-ade5-a7afadf10afe admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:16.553457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1130/2277] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:45:16 2022] POST /identity/v3/projects => generated 399 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:16.554807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94576aca-52dd-48ae-97d5-870d84c901cc None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:16.579234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94576aca-52dd-48ae-97d5-870d84c901cc None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:16.686947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60918e0-1d63-4ca0-bb31-a0ad293c4e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:16.687718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60918e0-1d63-4ca0-bb31-a0ad293c4e07 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:16.687997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60918e0-1d63-4ca0-bb31-a0ad293c4e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:16.688261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60918e0-1d63-4ca0-bb31-a0ad293c4e07 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:16.688894 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60918e0-1d63-4ca0-bb31-a0ad293c4e07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:16.691499 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60918e0-1d63-4ca0-bb31-a0ad293c4e07 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:16.808668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1148/2278] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:45:16 2022] POST /identity/v3/users => generated 306 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:16.808859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233d975-3cf9-406d-ade5-a7afadf10afe admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:16.819342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233d975-3cf9-406d-ade5-a7afadf10afe admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:16.824680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42b061-e7db-455b-9696-ce4b4ae5db35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:16.825470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42b061-e7db-455b-9696-ce4b4ae5db35 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:16.825863 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42b061-e7db-455b-9696-ce4b4ae5db35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:16.826231 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42b061-e7db-455b-9696-ce4b4ae5db35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:16.826951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42b061-e7db-455b-9696-ce4b4ae5db35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:16.828848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42b061-e7db-455b-9696-ce4b4ae5db35 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:16.846981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60918e0-1d63-4ca0-bb31-a0ad293c4e07 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:16.847278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1131/2279] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:16.859612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60918e0-1d63-4ca0-bb31-a0ad293c4e07 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:16.864657 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fae43-3d15-458a-872d-f54e70f1b827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:16.866003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fae43-3d15-458a-872d-f54e70f1b827 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:16.866003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fae43-3d15-458a-872d-f54e70f1b827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:16.866394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1fae43-3d15-458a-872d-f54e70f1b827 admin admin] PATH_INFO: `/v3/projects/e26290392b35498da0e793f4324d257d/users/57978a5a98d5415f9a73c6245be3d6ab/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:16.885937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fae43-3d15-458a-872d-f54e70f1b827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e26290392b35498da0e793f4324d257d, user_id=57978a5a98d5415f9a73c6245be3d6ab, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:16.889126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1fae43-3d15-458a-872d-f54e70f1b827 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:16.962929 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1149/2280] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:45:16 2022] PUT /identity/v3/projects/e26290392b35498da0e793f4324d257d/users/57978a5a98d5415f9a73c6245be3d6ab/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:45:16.966970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42b061-e7db-455b-9696-ce4b4ae5db35 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:16.979094 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42b061-e7db-455b-9696-ce4b4ae5db35 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:17.060230 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f708f-493c-4fb4-9b4d-24023f16c9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:17.062504 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f708f-493c-4fb4-9b4d-24023f16c9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:17.062504 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f708f-493c-4fb4-9b4d-24023f16c9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:17.062708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f708f-493c-4fb4-9b4d-24023f16c9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:17.063510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f708f-493c-4fb4-9b4d-24023f16c9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:17.065220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f708f-493c-4fb4-9b4d-24023f16c9f9 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:17.079546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1132/2281] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:17.080776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fae43-3d15-458a-872d-f54e70f1b827 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:17.087959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:17.095448 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1fae43-3d15-458a-872d-f54e70f1b827 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:17.100238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9960208-704d-43af-9a1d-1397d9c5b7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:17.100907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9960208-704d-43af-9a1d-1397d9c5b7a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:17.101183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9960208-704d-43af-9a1d-1397d9c5b7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:17.101452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9960208-704d-43af-9a1d-1397d9c5b7a2 admin admin] PATH_INFO: `/v3/projects/e26290392b35498da0e793f4324d257d/users/57978a5a98d5415f9a73c6245be3d6ab/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:17.103435 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9960208-704d-43af-9a1d-1397d9c5b7a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e26290392b35498da0e793f4324d257d, user_id=57978a5a98d5415f9a73c6245be3d6ab, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:17.106071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9960208-704d-43af-9a1d-1397d9c5b7a2 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:17.119767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1150/2282] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:45:17 2022] PUT /identity/v3/projects/e26290392b35498da0e793f4324d257d/users/57978a5a98d5415f9a73c6245be3d6ab/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:17.131337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742fbc8-3a64-400c-8223-46fe1806eff2 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:17.131337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742fbc8-3a64-400c-8223-46fe1806eff2 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:17.131546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742fbc8-3a64-400c-8223-46fe1806eff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:17.162389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c742fbc8-3a64-400c-8223-46fe1806eff2 None None] MFA Rules not processed for user `57978a5a98d5415f9a73c6245be3d6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:17.163628 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c742fbc8-3a64-400c-8223-46fe1806eff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:17.204325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:17.244838 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1133/2283] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:17 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:17.255758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9960208-704d-43af-9a1d-1397d9c5b7a2 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:17.257490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c9960208-704d-43af-9a1d-1397d9c5b7a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:17.323617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9960208-704d-43af-9a1d-1397d9c5b7a2 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:17.327442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302edaa9-c76f-4a31-849e-0848afd1a549 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:17.328054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302edaa9-c76f-4a31-849e-0848afd1a549 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:17.328332 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302edaa9-c76f-4a31-849e-0848afd1a549 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:17.328611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302edaa9-c76f-4a31-849e-0848afd1a549 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:17.360023 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302edaa9-c76f-4a31-849e-0848afd1a549 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:17.360023 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302edaa9-c76f-4a31-849e-0848afd1a549 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:17.371413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1151/2284] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:17 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:17.627627 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742fbc8-3a64-400c-8223-46fe1806eff2 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:17.634804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c742fbc8-3a64-400c-8223-46fe1806eff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:17.653038 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742fbc8-3a64-400c-8223-46fe1806eff2 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:17.658922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39acd4-6f10-48ba-b318-413029449bac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:17.658922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39acd4-6f10-48ba-b318-413029449bac service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:17.659422 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39acd4-6f10-48ba-b318-413029449bac service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:17.659831 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39acd4-6f10-48ba-b318-413029449bac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:17.783863 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39acd4-6f10-48ba-b318-413029449bac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:17.787768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d39acd4-6f10-48ba-b318-413029449bac service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:17.817197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1134/2285] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:17 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:23.100786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302edaa9-c76f-4a31-849e-0848afd1a549 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:23.213759 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302edaa9-c76f-4a31-849e-0848afd1a549 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:23.218786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:23.219507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:23.219782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:23.220057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] PATH_INFO: `/v3/users/483ec8865ec3471e9ac6867e91ff99f5` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:23.221263 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=483ec8865ec3471e9ac6867e91ff99f5)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:23.223999 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:23.460510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '483ec8865ec3471e9ac6867e91ff99f5'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:23.461175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '483ec8865ec3471e9ac6867e91ff99f5'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:23.525809 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '483ec8865ec3471e9ac6867e91ff99f5'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:23.527574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '483ec8865ec3471e9ac6867e91ff99f5'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:23.541443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '483ec8865ec3471e9ac6867e91ff99f5'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:23.582076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1152/2286] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:45:23 2022] DELETE /identity/v3/users/483ec8865ec3471e9ac6867e91ff99f5 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:23.583743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f4137-31ee-4cd9-8ace-258944d3a7b6 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:23.583743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f4137-31ee-4cd9-8ace-258944d3a7b6 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:23.583743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f4137-31ee-4cd9-8ace-258944d3a7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:23.607029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c53f4137-31ee-4cd9-8ace-258944d3a7b6 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:23.608354 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c53f4137-31ee-4cd9-8ace-258944d3a7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:23.678342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1135/2287] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:45:23.691153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:23.694018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:23.713207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:23.765813 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06abeff-bb50-4796-9c11-34bd34b19e9c admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:23.770185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfbbb4-5df2-40da-b827-578bf0018466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:23.770910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfbbb4-5df2-40da-b827-578bf0018466 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:23.771178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfbbb4-5df2-40da-b827-578bf0018466 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:23.771441 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bfbbb4-5df2-40da-b827-578bf0018466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:23.780926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bfbbb4-5df2-40da-b827-578bf0018466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:23.782431 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bfbbb4-5df2-40da-b827-578bf0018466 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:23.804039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1153/2288] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:24.225012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f4137-31ee-4cd9-8ace-258944d3a7b6 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:24.227137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c53f4137-31ee-4cd9-8ace-258944d3a7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:24.247101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f4137-31ee-4cd9-8ace-258944d3a7b6 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:24.252106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:24.252860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:24.253227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:24.253703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] PATH_INFO: `/v3/projects/bce073fbd9234c909f533c059fc949fe` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:24.254999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bce073fbd9234c909f533c059fc949fe)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:24.258188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:24.396839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bce073fbd9234c909f533c059fc949fe', 'request_id': 'req-56dfda3f-6f64-4d23-b494-97ffff5a142d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:24.403083 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bce073fbd9234c909f533c059fc949fe', 'request_id': 'req-56dfda3f-6f64-4d23-b494-97ffff5a142d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:24.404427 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:24.410247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bce073fbd9234c909f533c059fc949fe', 'request_id': 'req-56dfda3f-6f64-4d23-b494-97ffff5a142d'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:24.500596 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] The token cache is being invalidate because project bce073fbd9234c909f533c059fc949fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:45:24.500829 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:24.501489 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:24.503028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1136/2289] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:45:24 2022] DELETE /identity/v3/projects/bce073fbd9234c909f533c059fc949fe => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:24.508537 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfbbb4-5df2-40da-b827-578bf0018466 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:24.518300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bfbbb4-5df2-40da-b827-578bf0018466 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:24.579525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80333f8e-f27d-4971-93f5-6dde12add6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:24.580121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80333f8e-f27d-4971-93f5-6dde12add6c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:24.580402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80333f8e-f27d-4971-93f5-6dde12add6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:24.580681 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80333f8e-f27d-4971-93f5-6dde12add6c5 admin admin] PATH_INFO: `/v3/users/57978a5a98d5415f9a73c6245be3d6ab` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:24.581908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80333f8e-f27d-4971-93f5-6dde12add6c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57978a5a98d5415f9a73c6245be3d6ab)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:24.584286 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80333f8e-f27d-4971-93f5-6dde12add6c5 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:24.713805 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-80333f8e-f27d-4971-93f5-6dde12add6c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57978a5a98d5415f9a73c6245be3d6ab'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:24.714552 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-80333f8e-f27d-4971-93f5-6dde12add6c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57978a5a98d5415f9a73c6245be3d6ab'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:24.765510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-80333f8e-f27d-4971-93f5-6dde12add6c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57978a5a98d5415f9a73c6245be3d6ab'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:24.766193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-80333f8e-f27d-4971-93f5-6dde12add6c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57978a5a98d5415f9a73c6245be3d6ab'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:24.774624 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-80333f8e-f27d-4971-93f5-6dde12add6c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57978a5a98d5415f9a73c6245be3d6ab'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:24.792851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1154/2290] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:45:24 2022] DELETE /identity/v3/users/57978a5a98d5415f9a73c6245be3d6ab => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:25.464581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:25.466114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:25.479731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfda3f-6f64-4d23-b494-97ffff5a142d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:25.557788 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37027a83-d528-46f0-b32c-6fd139898601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:25.558301 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37027a83-d528-46f0-b32c-6fd139898601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:25.558581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37027a83-d528-46f0-b32c-6fd139898601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:25.558859 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37027a83-d528-46f0-b32c-6fd139898601 admin admin] PATH_INFO: `/v3/projects/e26290392b35498da0e793f4324d257d` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:25.560273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37027a83-d528-46f0-b32c-6fd139898601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e26290392b35498da0e793f4324d257d)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:25.562298 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37027a83-d528-46f0-b32c-6fd139898601 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:25.671738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-37027a83-d528-46f0-b32c-6fd139898601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e26290392b35498da0e793f4324d257d', 'request_id': 'req-37027a83-d528-46f0-b32c-6fd139898601'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:25.679870 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-37027a83-d528-46f0-b32c-6fd139898601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e26290392b35498da0e793f4324d257d', 'request_id': 'req-37027a83-d528-46f0-b32c-6fd139898601'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:25.686639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-37027a83-d528-46f0-b32c-6fd139898601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e26290392b35498da0e793f4324d257d', 'request_id': 'req-37027a83-d528-46f0-b32c-6fd139898601'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:25.765698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-37027a83-d528-46f0-b32c-6fd139898601 admin admin] The token cache is being invalidate because project e26290392b35498da0e793f4324d257d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:45:25.766061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-37027a83-d528-46f0-b32c-6fd139898601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:25.766713 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-37027a83-d528-46f0-b32c-6fd139898601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:25.768180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1137/2291] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:45:25 2022] DELETE /identity/v3/projects/e26290392b35498da0e793f4324d257d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:25.774787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a0009-5002-4bc4-b7d5-239a6a6818cd None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:25.775458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a0009-5002-4bc4-b7d5-239a6a6818cd None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:25.775458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a0009-5002-4bc4-b7d5-239a6a6818cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:25.797994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-678a0009-5002-4bc4-b7d5-239a6a6818cd None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:25.799278 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-678a0009-5002-4bc4-b7d5-239a6a6818cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:25.836103 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:25.868898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1155/2292] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:25.881167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef8fb3-ed47-4d07-bcd1-16bc1e273060 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:25.881648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef8fb3-ed47-4d07-bcd1-16bc1e273060 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:25.882161 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef8fb3-ed47-4d07-bcd1-16bc1e273060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:25.912633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-b6ef8fb3-ed47-4d07-bcd1-16bc1e273060 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:25.913870 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b6ef8fb3-ed47-4d07-bcd1-16bc1e273060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:25.925779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1138/2293] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:25.930966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a0009-5002-4bc4-b7d5-239a6a6818cd None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:25.943321 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a0009-5002-4bc4-b7d5-239a6a6818cd None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:25.948359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc973372-64ac-4987-9309-a166ee25ccdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:25.949209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973372-64ac-4987-9309-a166ee25ccdd admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:25.949487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973372-64ac-4987-9309-a166ee25ccdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:25.949755 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc973372-64ac-4987-9309-a166ee25ccdd admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:25.950685 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc973372-64ac-4987-9309-a166ee25ccdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:45:25.950990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc973372-64ac-4987-9309-a166ee25ccdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:25.952580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc973372-64ac-4987-9309-a166ee25ccdd admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:25.965408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1156/2294] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:45:25.969620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef8fb3-ed47-4d07-bcd1-16bc1e273060 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:25.979409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef8fb3-ed47-4d07-bcd1-16bc1e273060 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:25.984407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9df36-66bd-4612-a44a-d5f8ff30c82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:25.985044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9df36-66bd-4612-a44a-d5f8ff30c82a admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:25.985333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9df36-66bd-4612-a44a-d5f8ff30c82a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:25.985648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c9df36-66bd-4612-a44a-d5f8ff30c82a admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:25.986488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9df36-66bd-4612-a44a-d5f8ff30c82a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:25.988907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c9df36-66bd-4612-a44a-d5f8ff30c82a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:26.093096 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1139/2295] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:45:25 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:26.097403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc973372-64ac-4987-9309-a166ee25ccdd admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:26.108439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc973372-64ac-4987-9309-a166ee25ccdd admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:26.174301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f3f90-8084-4a0e-91f8-8b48dd30d938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:26.174921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f3f90-8084-4a0e-91f8-8b48dd30d938 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:26.175222 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f3f90-8084-4a0e-91f8-8b48dd30d938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:26.175462 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f3f90-8084-4a0e-91f8-8b48dd30d938 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:26.176102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f3f90-8084-4a0e-91f8-8b48dd30d938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:26.178185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f3f90-8084-4a0e-91f8-8b48dd30d938 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:26.258828 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1157/2296] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:45:26 2022] POST /identity/v3/users => generated 284 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:26.262348 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9df36-66bd-4612-a44a-d5f8ff30c82a admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:26.274202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c9df36-66bd-4612-a44a-d5f8ff30c82a admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:26.280476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09344614-b33b-4053-90f2-da7387a77e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:26.282051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09344614-b33b-4053-90f2-da7387a77e8d admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:26.282051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09344614-b33b-4053-90f2-da7387a77e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:26.282051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09344614-b33b-4053-90f2-da7387a77e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:26.282396 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09344614-b33b-4053-90f2-da7387a77e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:26.283952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09344614-b33b-4053-90f2-da7387a77e8d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:26.295228 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1140/2297] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:26.299091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f3f90-8084-4a0e-91f8-8b48dd30d938 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:26.309733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f3f90-8084-4a0e-91f8-8b48dd30d938 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:26.314687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35faa76-abb5-4bc8-94e7-22a8c164f605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:26.315280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35faa76-abb5-4bc8-94e7-22a8c164f605 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:26.315560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35faa76-abb5-4bc8-94e7-22a8c164f605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:26.315833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35faa76-abb5-4bc8-94e7-22a8c164f605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:26.319020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35faa76-abb5-4bc8-94e7-22a8c164f605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:26.319020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35faa76-abb5-4bc8-94e7-22a8c164f605 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:26.330557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1158/2298] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:26.336748 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09344614-b33b-4053-90f2-da7387a77e8d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:26.351971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09344614-b33b-4053-90f2-da7387a77e8d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:26.357851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b6b1f-f1fd-4246-b6b0-04d38c6458d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:26.357851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b6b1f-f1fd-4246-b6b0-04d38c6458d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:26.361245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b6b1f-f1fd-4246-b6b0-04d38c6458d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:26.361245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b6b1f-f1fd-4246-b6b0-04d38c6458d7 admin admin] PATH_INFO: `/v3/projects/de2cc0b723f64229825c0e3aff29438d/users/8eafaec8780a49e595df4474c0dfc0b4/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:26.375021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b6b1f-f1fd-4246-b6b0-04d38c6458d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de2cc0b723f64229825c0e3aff29438d, user_id=8eafaec8780a49e595df4474c0dfc0b4, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:26.377668 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2b6b1f-f1fd-4246-b6b0-04d38c6458d7 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:26.498217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1141/2299] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:45:26 2022] PUT /identity/v3/projects/de2cc0b723f64229825c0e3aff29438d/users/8eafaec8780a49e595df4474c0dfc0b4/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:26.508590 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5296d0f-6a9a-4435-ad88-7a7382aea649 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:26.508868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5296d0f-6a9a-4435-ad88-7a7382aea649 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:26.509137 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5296d0f-6a9a-4435-ad88-7a7382aea649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:26.540341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-b5296d0f-6a9a-4435-ad88-7a7382aea649 None None] MFA Rules not processed for user `8eafaec8780a49e595df4474c0dfc0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:26.541555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b5296d0f-6a9a-4435-ad88-7a7382aea649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:26.614243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1159/2300] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:26.619434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b6b1f-f1fd-4246-b6b0-04d38c6458d7 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:26.634865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:26.687459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b6b1f-f1fd-4246-b6b0-04d38c6458d7 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:26.692289 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c019c-df44-4b4a-9364-bf235d49248c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:26.693456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c019c-df44-4b4a-9364-bf235d49248c admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:26.693813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c019c-df44-4b4a-9364-bf235d49248c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:26.694198 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c019c-df44-4b4a-9364-bf235d49248c admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:26.694720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c019c-df44-4b4a-9364-bf235d49248c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:26.696660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c019c-df44-4b4a-9364-bf235d49248c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:26.813915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1142/2301] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:45:26 2022] POST /identity/v3/projects => generated 367 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:26.818646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5296d0f-6a9a-4435-ad88-7a7382aea649 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:26.829381 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5296d0f-6a9a-4435-ad88-7a7382aea649 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:26.857913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:26.891626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909aaa8d-71dc-4501-93a8-78cbe6fa1cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:26.892237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909aaa8d-71dc-4501-93a8-78cbe6fa1cbe admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:26.892510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909aaa8d-71dc-4501-93a8-78cbe6fa1cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:26.892780 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909aaa8d-71dc-4501-93a8-78cbe6fa1cbe admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:26.893429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909aaa8d-71dc-4501-93a8-78cbe6fa1cbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:26.897170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909aaa8d-71dc-4501-93a8-78cbe6fa1cbe admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:26.997868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1160/2302] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:45:26 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:26.999131 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c019c-df44-4b4a-9364-bf235d49248c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:27.013345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c019c-df44-4b4a-9364-bf235d49248c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:27.021915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffd8b5-70b2-476a-9f25-5a5d984f1c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:27.022695 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffd8b5-70b2-476a-9f25-5a5d984f1c27 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:27.023034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffd8b5-70b2-476a-9f25-5a5d984f1c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:27.023374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffd8b5-70b2-476a-9f25-5a5d984f1c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:27.024095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffd8b5-70b2-476a-9f25-5a5d984f1c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:27.025728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffd8b5-70b2-476a-9f25-5a5d984f1c27 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:27.038797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1143/2303] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:27.042632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909aaa8d-71dc-4501-93a8-78cbe6fa1cbe admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:27.055212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909aaa8d-71dc-4501-93a8-78cbe6fa1cbe admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:27.064510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7fff2-884d-4f1a-a42f-0de8b09718fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:27.065225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7fff2-884d-4f1a-a42f-0de8b09718fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:27.065494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7fff2-884d-4f1a-a42f-0de8b09718fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:27.065761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7fff2-884d-4f1a-a42f-0de8b09718fa admin admin] PATH_INFO: `/v3/projects/fea17d8bf8e3471abb15e1281fec96c8/users/50f3231f539b4adea438f718ba51fc22/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:27.082645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7fff2-884d-4f1a-a42f-0de8b09718fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea17d8bf8e3471abb15e1281fec96c8, user_id=50f3231f539b4adea438f718ba51fc22, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:27.085619 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7fff2-884d-4f1a-a42f-0de8b09718fa admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:27.194267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1161/2304] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:45:27 2022] PUT /identity/v3/projects/fea17d8bf8e3471abb15e1281fec96c8/users/50f3231f539b4adea438f718ba51fc22/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:27.199452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffd8b5-70b2-476a-9f25-5a5d984f1c27 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:27.214214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffd8b5-70b2-476a-9f25-5a5d984f1c27 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:27.297934 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda9b5b7-58e3-4107-a8c8-ac423245c22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:27.298664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9b5b7-58e3-4107-a8c8-ac423245c22d admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:27.299056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9b5b7-58e3-4107-a8c8-ac423245c22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:27.299425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda9b5b7-58e3-4107-a8c8-ac423245c22d admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:27.300170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9b5b7-58e3-4107-a8c8-ac423245c22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:27.301791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda9b5b7-58e3-4107-a8c8-ac423245c22d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:27.313221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1144/2305] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:27.317438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7fff2-884d-4f1a-a42f-0de8b09718fa admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:27.330021 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7fff2-884d-4f1a-a42f-0de8b09718fa admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:27.339271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257b469-2a20-4a04-ad56-5c50ec5f3077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:27.339897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257b469-2a20-4a04-ad56-5c50ec5f3077 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:27.340172 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257b469-2a20-4a04-ad56-5c50ec5f3077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:27.340441 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257b469-2a20-4a04-ad56-5c50ec5f3077 admin admin] PATH_INFO: `/v3/projects/fea17d8bf8e3471abb15e1281fec96c8/users/50f3231f539b4adea438f718ba51fc22/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:27.342477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257b469-2a20-4a04-ad56-5c50ec5f3077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea17d8bf8e3471abb15e1281fec96c8, user_id=50f3231f539b4adea438f718ba51fc22, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:27.346357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257b469-2a20-4a04-ad56-5c50ec5f3077 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:27.366426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1162/2306] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:45:27 2022] PUT /identity/v3/projects/fea17d8bf8e3471abb15e1281fec96c8/users/50f3231f539b4adea438f718ba51fc22/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:27.371281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51475197-5e62-48a3-918e-ee2736fef6ba None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:27.371590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51475197-5e62-48a3-918e-ee2736fef6ba None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:27.371860 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51475197-5e62-48a3-918e-ee2736fef6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:27.405955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-51475197-5e62-48a3-918e-ee2736fef6ba None None] MFA Rules not processed for user `50f3231f539b4adea438f718ba51fc22`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:27.407183 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-51475197-5e62-48a3-918e-ee2736fef6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:27.491864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1145/2307] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:27.503817 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257b469-2a20-4a04-ad56-5c50ec5f3077 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:27.506308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-6257b469-2a20-4a04-ad56-5c50ec5f3077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:27.573660 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257b469-2a20-4a04-ad56-5c50ec5f3077 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:27.577493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d43b5-97ca-48d4-ad64-62933562c233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:27.578213 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d43b5-97ca-48d4-ad64-62933562c233 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:27.578549 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d43b5-97ca-48d4-ad64-62933562c233 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:27.578988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d43b5-97ca-48d4-ad64-62933562c233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:27.589218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d43b5-97ca-48d4-ad64-62933562c233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:27.590677 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d43b5-97ca-48d4-ad64-62933562c233 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:27.600274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1163/2308] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:27.822834 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51475197-5e62-48a3-918e-ee2736fef6ba None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:27.835319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51475197-5e62-48a3-918e-ee2736fef6ba None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:27.843470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14c0e3-e855-4df2-a155-d4c72c9f8ba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:27.847887 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14c0e3-e855-4df2-a155-d4c72c9f8ba0 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:27.847887 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14c0e3-e855-4df2-a155-d4c72c9f8ba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:27.847887 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14c0e3-e855-4df2-a155-d4c72c9f8ba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:27.852039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:27.928144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14c0e3-e855-4df2-a155-d4c72c9f8ba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:27.931917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14c0e3-e855-4df2-a155-d4c72c9f8ba0 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:27.940495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1146/2309] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:30.582215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841664f7-622a-4abb-9f19-2046832346fe None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:30.582706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841664f7-622a-4abb-9f19-2046832346fe None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:30.582990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841664f7-622a-4abb-9f19-2046832346fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:30.607452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-841664f7-622a-4abb-9f19-2046832346fe None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:30.609972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-841664f7-622a-4abb-9f19-2046832346fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:30.679301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1164/2310] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:45:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:30.723046 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14c0e3-e855-4df2-a155-d4c72c9f8ba0 service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:30.724709 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6e14c0e3-e855-4df2-a155-d4c72c9f8ba0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:30.799715 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14c0e3-e855-4df2-a155-d4c72c9f8ba0 service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:30.805687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83f609-f0ce-47e2-a2c7-31656bff4dbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:30.806757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83f609-f0ce-47e2-a2c7-31656bff4dbe service swift] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:30.807084 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83f609-f0ce-47e2-a2c7-31656bff4dbe service swift] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:30.807349 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83f609-f0ce-47e2-a2c7-31656bff4dbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:30.818217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83f609-f0ce-47e2-a2c7-31656bff4dbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:30.819825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83f609-f0ce-47e2-a2c7-31656bff4dbe service swift] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:30.828586 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1147/2311] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:45:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:45:35.295853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841664f7-622a-4abb-9f19-2046832346fe None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:35.332181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:35.381659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841664f7-622a-4abb-9f19-2046832346fe None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:35.386782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:35.389211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:35.389211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:35.389211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] PATH_INFO: `/v3/users/8eafaec8780a49e595df4474c0dfc0b4` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:35.389211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eafaec8780a49e595df4474c0dfc0b4)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:35.391217 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:35.588878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eafaec8780a49e595df4474c0dfc0b4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:35.596223 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eafaec8780a49e595df4474c0dfc0b4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:35.695768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eafaec8780a49e595df4474c0dfc0b4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:35.696597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eafaec8780a49e595df4474c0dfc0b4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:35.711527 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eafaec8780a49e595df4474c0dfc0b4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:35.753628 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1165/2312] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:45:35 2022] DELETE /identity/v3/users/8eafaec8780a49e595df4474c0dfc0b4 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:35.759669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7e29-e1d3-4c04-a302-b484cbca64c6 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:35.760992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7e29-e1d3-4c04-a302-b484cbca64c6 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:35.761550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7e29-e1d3-4c04-a302-b484cbca64c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:35.806662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-dffc7e29-e1d3-4c04-a302-b484cbca64c6 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:35.809811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-dffc7e29-e1d3-4c04-a302-b484cbca64c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:35.917661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1148/2313] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:35.933864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:35.933864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:36.058530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66640b19-ea3f-4356-870c-5f5fdebf4e71 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:36.063768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0b9c9-a2c1-4e01-b415-bbe2ab478582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:36.064589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0b9c9-a2c1-4e01-b415-bbe2ab478582 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:36.064918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0b9c9-a2c1-4e01-b415-bbe2ab478582 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:36.065337 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0b9c9-a2c1-4e01-b415-bbe2ab478582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:36.078597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0b9c9-a2c1-4e01-b415-bbe2ab478582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:36.080185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0b9c9-a2c1-4e01-b415-bbe2ab478582 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:36.102018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1166/2314] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:36.588074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7e29-e1d3-4c04-a302-b484cbca64c6 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:36.589172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-dffc7e29-e1d3-4c04-a302-b484cbca64c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:36.608021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7e29-e1d3-4c04-a302-b484cbca64c6 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:36.617593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:36.618359 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:36.622991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:36.622991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] PATH_INFO: `/v3/projects/de2cc0b723f64229825c0e3aff29438d` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:36.622991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] RBAC: Authorizing `identity:delete_project(project_id=de2cc0b723f64229825c0e3aff29438d)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:36.622991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:36.723642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:36.741481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de2cc0b723f64229825c0e3aff29438d', 'request_id': 'req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:36.748677 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de2cc0b723f64229825c0e3aff29438d', 'request_id': 'req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:36.757066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de2cc0b723f64229825c0e3aff29438d', 'request_id': 'req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:36.841802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] The token cache is being invalidate because project de2cc0b723f64229825c0e3aff29438d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:45:36.842461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:36.845855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:36.848239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1149/2315] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:45:36 2022] DELETE /identity/v3/projects/de2cc0b723f64229825c0e3aff29438d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:36.852665 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0b9c9-a2c1-4e01-b415-bbe2ab478582 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:36.867393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0b9c9-a2c1-4e01-b415-bbe2ab478582 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:36.949327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:36.963908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763ad0d-1304-4e2d-b406-5b34db97b2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:36.964856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763ad0d-1304-4e2d-b406-5b34db97b2d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:36.965281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763ad0d-1304-4e2d-b406-5b34db97b2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:36.965673 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763ad0d-1304-4e2d-b406-5b34db97b2d8 admin admin] PATH_INFO: `/v3/users/50f3231f539b4adea438f718ba51fc22` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:36.967035 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763ad0d-1304-4e2d-b406-5b34db97b2d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f3231f539b4adea438f718ba51fc22)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:36.969070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763ad0d-1304-4e2d-b406-5b34db97b2d8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:37.181262 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8763ad0d-1304-4e2d-b406-5b34db97b2d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f3231f539b4adea438f718ba51fc22'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:37.182566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8763ad0d-1304-4e2d-b406-5b34db97b2d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f3231f539b4adea438f718ba51fc22'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:37.257918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8763ad0d-1304-4e2d-b406-5b34db97b2d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f3231f539b4adea438f718ba51fc22'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:37.259136 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8763ad0d-1304-4e2d-b406-5b34db97b2d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f3231f539b4adea438f718ba51fc22'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:37.276620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8763ad0d-1304-4e2d-b406-5b34db97b2d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f3231f539b4adea438f718ba51fc22'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:37.303140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1167/2316] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:45:36 2022] DELETE /identity/v3/users/50f3231f539b4adea438f718ba51fc22 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:38.093619 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:38.097205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:38.117691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35b5f1-cba8-409f-9ef5-b2db14a24fde admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:38.230180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:38.233542 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:38.233947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:38.234301 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] PATH_INFO: `/v3/projects/fea17d8bf8e3471abb15e1281fec96c8` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:38.235714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea17d8bf8e3471abb15e1281fec96c8)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:38.237694 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:38.404909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea17d8bf8e3471abb15e1281fec96c8', 'request_id': 'req-b47ee405-8747-4213-9dae-b44a7b36a30c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:38.417264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea17d8bf8e3471abb15e1281fec96c8', 'request_id': 'req-b47ee405-8747-4213-9dae-b44a7b36a30c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:38.426447 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea17d8bf8e3471abb15e1281fec96c8', 'request_id': 'req-b47ee405-8747-4213-9dae-b44a7b36a30c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:38.523623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] The token cache is being invalidate because project fea17d8bf8e3471abb15e1281fec96c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:45:38.523971 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:38.525524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:38.531266 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1150/2317] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:45:38 2022] DELETE /identity/v3/projects/fea17d8bf8e3471abb15e1281fec96c8 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:38.536997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b205e41-7be8-4edc-99fc-c9b1483347f7 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:38.537532 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b205e41-7be8-4edc-99fc-c9b1483347f7 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:38.537622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b205e41-7be8-4edc-99fc-c9b1483347f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:38.562472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-7b205e41-7be8-4edc-99fc-c9b1483347f7 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:38.564133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7b205e41-7be8-4edc-99fc-c9b1483347f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:38.611685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:38.613782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:38.664812 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1168/2318] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:38.666901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b679a-1657-4b35-956b-1966c7be6ae8 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:38.667260 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b679a-1657-4b35-956b-1966c7be6ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:38.667617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b679a-1657-4b35-956b-1966c7be6ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:38.693428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ee405-8747-4213-9dae-b44a7b36a30c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:38.697232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed64d7-8d25-4ce6-a518-b01f7e8697e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:38.697901 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed64d7-8d25-4ce6-a518-b01f7e8697e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:38.698142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed64d7-8d25-4ce6-a518-b01f7e8697e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:38.699055 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed64d7-8d25-4ce6-a518-b01f7e8697e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:38.700562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-360b679a-1657-4b35-956b-1966c7be6ae8 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:38.700854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-55ed64d7-8d25-4ce6-a518-b01f7e8697e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:38.702360 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-360b679a-1657-4b35-956b-1966c7be6ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:38.706433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:38.712543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1169/2319] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:38.717206 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b679a-1657-4b35-956b-1966c7be6ae8 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:38.736266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b679a-1657-4b35-956b-1966c7be6ae8 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:38.746049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38a918-dbf2-4cbc-aa27-03b7cf0a075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:38.747539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38a918-dbf2-4cbc-aa27-03b7cf0a075c admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:38.748169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38a918-dbf2-4cbc-aa27-03b7cf0a075c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:38.749293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38a918-dbf2-4cbc-aa27-03b7cf0a075c admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:38.750139 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38a918-dbf2-4cbc-aa27-03b7cf0a075c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:45:38.750695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38a918-dbf2-4cbc-aa27-03b7cf0a075c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:38.752789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38a918-dbf2-4cbc-aa27-03b7cf0a075c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:38.772733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1170/2320] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:45:38.782736 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38a918-dbf2-4cbc-aa27-03b7cf0a075c admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:38.787415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed64d7-8d25-4ce6-a518-b01f7e8697e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:38.789375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed64d7-8d25-4ce6-a518-b01f7e8697e8 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:38.795617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38a918-dbf2-4cbc-aa27-03b7cf0a075c admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:38.800582 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8456ec-7856-4d00-8f53-e2eaf0c7fed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:38.801283 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8456ec-7856-4d00-8f53-e2eaf0c7fed2 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:38.802108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8456ec-7856-4d00-8f53-e2eaf0c7fed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:38.802379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8456ec-7856-4d00-8f53-e2eaf0c7fed2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:38.803008 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8456ec-7856-4d00-8f53-e2eaf0c7fed2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:38.804889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8456ec-7856-4d00-8f53-e2eaf0c7fed2 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:38.815698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1151/2321] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:38.893081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1171/2322] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:45:38 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:38.893651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed64d7-8d25-4ce6-a518-b01f7e8697e8 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:38.907551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed64d7-8d25-4ce6-a518-b01f7e8697e8 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:38.978589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f17a9-eb7d-4d19-ad27-90f991eb3d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:38.979255 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f17a9-eb7d-4d19-ad27-90f991eb3d72 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:38.979701 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f17a9-eb7d-4d19-ad27-90f991eb3d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:38.981059 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0f17a9-eb7d-4d19-ad27-90f991eb3d72 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:38.981992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f17a9-eb7d-4d19-ad27-90f991eb3d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:38.985296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0f17a9-eb7d-4d19-ad27-90f991eb3d72 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:39.057529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1152/2323] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:45:38 2022] POST /identity/v3/users => generated 294 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:45:39.062032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8456ec-7856-4d00-8f53-e2eaf0c7fed2 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:39.074855 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8456ec-7856-4d00-8f53-e2eaf0c7fed2 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:39.081810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70806e32-2eb7-4176-9d78-a4d1d939aee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:39.086358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70806e32-2eb7-4176-9d78-a4d1d939aee4 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:39.086358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70806e32-2eb7-4176-9d78-a4d1d939aee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:39.086358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70806e32-2eb7-4176-9d78-a4d1d939aee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:39.086358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70806e32-2eb7-4176-9d78-a4d1d939aee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:39.087567 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70806e32-2eb7-4176-9d78-a4d1d939aee4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:39.101550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1172/2324] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:39.110978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f17a9-eb7d-4d19-ad27-90f991eb3d72 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:39.135483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0f17a9-eb7d-4d19-ad27-90f991eb3d72 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:39.149526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ac7a3-8b95-4640-936b-e8debbe60334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:39.150274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ac7a3-8b95-4640-936b-e8debbe60334 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:39.150614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ac7a3-8b95-4640-936b-e8debbe60334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:39.150955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ac7a3-8b95-4640-936b-e8debbe60334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:39.151688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ac7a3-8b95-4640-936b-e8debbe60334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:39.153618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ac7a3-8b95-4640-936b-e8debbe60334 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:39.184984 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1153/2325] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:39.186061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70806e32-2eb7-4176-9d78-a4d1d939aee4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:39.201515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70806e32-2eb7-4176-9d78-a4d1d939aee4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:39.217655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1f824-c7fc-47ed-aa08-fbfce4683fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:39.217655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1f824-c7fc-47ed-aa08-fbfce4683fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:39.217655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1f824-c7fc-47ed-aa08-fbfce4683fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:39.217655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1f824-c7fc-47ed-aa08-fbfce4683fcf admin admin] PATH_INFO: `/v3/projects/2529d2b6eea44450a592ef8e7a8b6c60/users/cf1d9fb75e5e487485454d0f8bc91da6/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:39.231753 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1f824-c7fc-47ed-aa08-fbfce4683fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2529d2b6eea44450a592ef8e7a8b6c60, user_id=cf1d9fb75e5e487485454d0f8bc91da6, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:39.236341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1f824-c7fc-47ed-aa08-fbfce4683fcf admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:39.240161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:39.328511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1173/2326] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:45:39 2022] PUT /identity/v3/projects/2529d2b6eea44450a592ef8e7a8b6c60/users/cf1d9fb75e5e487485454d0f8bc91da6/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:39.359903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b74fdb-b15d-460c-a310-806d1806db16 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:39.361076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b74fdb-b15d-460c-a310-806d1806db16 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:39.361432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b74fdb-b15d-460c-a310-806d1806db16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:39.398626 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-a9b74fdb-b15d-460c-a310-806d1806db16 None None] MFA Rules not processed for user `cf1d9fb75e5e487485454d0f8bc91da6`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:39.400048 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a9b74fdb-b15d-460c-a310-806d1806db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:39.485419 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1154/2327] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:39.491706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1f824-c7fc-47ed-aa08-fbfce4683fcf admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:39.579374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1f824-c7fc-47ed-aa08-fbfce4683fcf admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:39.587086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6955afa-6839-4020-831c-fd827e8109e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:39.587866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6955afa-6839-4020-831c-fd827e8109e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:39.588362 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6955afa-6839-4020-831c-fd827e8109e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:39.588724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6955afa-6839-4020-831c-fd827e8109e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:39.589457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6955afa-6839-4020-831c-fd827e8109e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:39.592580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6955afa-6839-4020-831c-fd827e8109e6 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:39.760925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1174/2328] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:45:39 2022] POST /identity/v3/projects => generated 385 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:45:39.765051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b74fdb-b15d-460c-a310-806d1806db16 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:39.786843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b74fdb-b15d-460c-a310-806d1806db16 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:39.795119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:39.887683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca785ad-a631-412b-9d8b-f3e085241183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:39.888328 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca785ad-a631-412b-9d8b-f3e085241183 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:39.888608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca785ad-a631-412b-9d8b-f3e085241183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:39.889079 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca785ad-a631-412b-9d8b-f3e085241183 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:39.889505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca785ad-a631-412b-9d8b-f3e085241183 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:39.891466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca785ad-a631-412b-9d8b-f3e085241183 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:40.001865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1155/2329] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:45:39 2022] POST /identity/v3/users => generated 299 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:40.007646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6955afa-6839-4020-831c-fd827e8109e6 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:40.022189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6955afa-6839-4020-831c-fd827e8109e6 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:40.027617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15411d-28d4-41ba-96d1-e86573f3dcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:40.028607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15411d-28d4-41ba-96d1-e86573f3dcd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:40.029070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15411d-28d4-41ba-96d1-e86573f3dcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:40.029337 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15411d-28d4-41ba-96d1-e86573f3dcd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:40.030114 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15411d-28d4-41ba-96d1-e86573f3dcd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:40.032182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f15411d-28d4-41ba-96d1-e86573f3dcd9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:40.049459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1175/2330] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:40.050790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca785ad-a631-412b-9d8b-f3e085241183 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:40.062537 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca785ad-a631-412b-9d8b-f3e085241183 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:40.069110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91e649-2f86-42af-93a2-67389bda86bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:40.069914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91e649-2f86-42af-93a2-67389bda86bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:40.070271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91e649-2f86-42af-93a2-67389bda86bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:40.070727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91e649-2f86-42af-93a2-67389bda86bd admin admin] PATH_INFO: `/v3/projects/ec76534c9c244880b7710263c3cff857/users/e68c038d010f472bb0626544a4ac8232/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:40.098327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91e649-2f86-42af-93a2-67389bda86bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec76534c9c244880b7710263c3cff857, user_id=e68c038d010f472bb0626544a4ac8232, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:40.104190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91e649-2f86-42af-93a2-67389bda86bd admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:40.193069 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1156/2331] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:45:40 2022] PUT /identity/v3/projects/ec76534c9c244880b7710263c3cff857/users/e68c038d010f472bb0626544a4ac8232/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:45:40.193626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15411d-28d4-41ba-96d1-e86573f3dcd9 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:40.213648 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f15411d-28d4-41ba-96d1-e86573f3dcd9 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:40.282259 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc04ba-f1c9-4a78-9e59-b76cb067681e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:40.283436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc04ba-f1c9-4a78-9e59-b76cb067681e admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:40.283867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc04ba-f1c9-4a78-9e59-b76cb067681e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:40.284299 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc04ba-f1c9-4a78-9e59-b76cb067681e admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:40.285108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc04ba-f1c9-4a78-9e59-b76cb067681e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:40.287610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bc04ba-f1c9-4a78-9e59-b76cb067681e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:40.310639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91e649-2f86-42af-93a2-67389bda86bd admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:40.328418 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1176/2332] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:45:40.340144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91e649-2f86-42af-93a2-67389bda86bd admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:40.355218 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac895e2-ff30-4702-9ddf-68dd3f664dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:40.355913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac895e2-ff30-4702-9ddf-68dd3f664dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:40.356180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac895e2-ff30-4702-9ddf-68dd3f664dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:40.356447 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac895e2-ff30-4702-9ddf-68dd3f664dd5 admin admin] PATH_INFO: `/v3/projects/ec76534c9c244880b7710263c3cff857/users/e68c038d010f472bb0626544a4ac8232/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:40.360099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac895e2-ff30-4702-9ddf-68dd3f664dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec76534c9c244880b7710263c3cff857, user_id=e68c038d010f472bb0626544a4ac8232, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:40.365160 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac895e2-ff30-4702-9ddf-68dd3f664dd5 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:40.408689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b078df1-496b-4d6e-81d2-d5a24e6094ab None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:40.409050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b078df1-496b-4d6e-81d2-d5a24e6094ab None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:40.409255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b078df1-496b-4d6e-81d2-d5a24e6094ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:40.413088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1157/2333] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:45:40 2022] PUT /identity/v3/projects/ec76534c9c244880b7710263c3cff857/users/e68c038d010f472bb0626544a4ac8232/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:40.463218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-7b078df1-496b-4d6e-81d2-d5a24e6094ab None None] MFA Rules not processed for user `e68c038d010f472bb0626544a4ac8232`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:40.470017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7b078df1-496b-4d6e-81d2-d5a24e6094ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:40.561645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:40.591664 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1177/2334] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:40.605232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac895e2-ff30-4702-9ddf-68dd3f664dd5 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:40.607303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5ac895e2-ff30-4702-9ddf-68dd3f664dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:40.708137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac895e2-ff30-4702-9ddf-68dd3f664dd5 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:40.712230 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42015ea6-0c4f-440c-a69a-9fd359f1a632 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:40.713103 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42015ea6-0c4f-440c-a69a-9fd359f1a632 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:40.713586 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42015ea6-0c4f-440c-a69a-9fd359f1a632 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:40.714162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42015ea6-0c4f-440c-a69a-9fd359f1a632 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:40.726959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42015ea6-0c4f-440c-a69a-9fd359f1a632 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:40.728591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42015ea6-0c4f-440c-a69a-9fd359f1a632 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:40.750535 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1158/2335] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:45:42.182142 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b078df1-496b-4d6e-81d2-d5a24e6094ab None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:42.260180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b078df1-496b-4d6e-81d2-d5a24e6094ab None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:42.265849 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:42.266543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:42.266824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:42.267084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] PATH_INFO: `/v3/users/cf1d9fb75e5e487485454d0f8bc91da6` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:42.268308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1d9fb75e5e487485454d0f8bc91da6)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:42.271923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:42.463536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1d9fb75e5e487485454d0f8bc91da6'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:42.464644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1d9fb75e5e487485454d0f8bc91da6'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:42.537879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1d9fb75e5e487485454d0f8bc91da6'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:42.539535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1d9fb75e5e487485454d0f8bc91da6'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:42.551724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1d9fb75e5e487485454d0f8bc91da6'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:42.587031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1178/2336] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:45:42 2022] DELETE /identity/v3/users/cf1d9fb75e5e487485454d0f8bc91da6 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:42.591790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c983c6-559e-4ef3-92a6-21cc24f0dfe6 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:42.592449 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c983c6-559e-4ef3-92a6-21cc24f0dfe6 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:42.592449 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c983c6-559e-4ef3-92a6-21cc24f0dfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:42.614572 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-d6c983c6-559e-4ef3-92a6-21cc24f0dfe6 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:42.615922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d6c983c6-559e-4ef3-92a6-21cc24f0dfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:42.693838 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1159/2337] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:45:42.703632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:42.709758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:42.776503 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc3a25-99df-4df8-8a77-61abed07904b admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:42.781592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d0f05-7a8b-45aa-9855-576a1bbfbdfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:42.782237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d0f05-7a8b-45aa-9855-576a1bbfbdfe service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:42.782639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d0f05-7a8b-45aa-9855-576a1bbfbdfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:42.783029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d0f05-7a8b-45aa-9855-576a1bbfbdfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:42.795618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d0f05-7a8b-45aa-9855-576a1bbfbdfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:42.797573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d0f05-7a8b-45aa-9855-576a1bbfbdfe service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:42.810791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1179/2338] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:45:43.455213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c983c6-559e-4ef3-92a6-21cc24f0dfe6 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:43.456660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d6c983c6-559e-4ef3-92a6-21cc24f0dfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:43.466389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:43.472037 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c983c6-559e-4ef3-92a6-21cc24f0dfe6 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:43.479086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:43.479855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:43.480134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:43.482636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] PATH_INFO: `/v3/projects/2529d2b6eea44450a592ef8e7a8b6c60` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:43.482636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2529d2b6eea44450a592ef8e7a8b6c60)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:43.493077 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:43.654548 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2529d2b6eea44450a592ef8e7a8b6c60', 'request_id': 'req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:43.661431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2529d2b6eea44450a592ef8e7a8b6c60', 'request_id': 'req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:43.668925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2529d2b6eea44450a592ef8e7a8b6c60', 'request_id': 'req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:43.758970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] The token cache is being invalidate because project 2529d2b6eea44450a592ef8e7a8b6c60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:45:43.759256 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:43.759894 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:43.761360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1160/2339] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:45:43 2022] DELETE /identity/v3/projects/2529d2b6eea44450a592ef8e7a8b6c60 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:43.765491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d0f05-7a8b-45aa-9855-576a1bbfbdfe service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:43.775296 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d0f05-7a8b-45aa-9855-576a1bbfbdfe service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:43.783240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:43.832647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f9780-79a0-4753-a10c-f24958d2c0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:43.833289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f9780-79a0-4753-a10c-f24958d2c0a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:43.833554 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f9780-79a0-4753-a10c-f24958d2c0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:43.833817 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f9780-79a0-4753-a10c-f24958d2c0a6 admin admin] PATH_INFO: `/v3/users/e68c038d010f472bb0626544a4ac8232` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:43.834954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f9780-79a0-4753-a10c-f24958d2c0a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e68c038d010f472bb0626544a4ac8232)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:43.836891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191f9780-79a0-4753-a10c-f24958d2c0a6 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:43.985356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-191f9780-79a0-4753-a10c-f24958d2c0a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e68c038d010f472bb0626544a4ac8232'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:43.986147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-191f9780-79a0-4753-a10c-f24958d2c0a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e68c038d010f472bb0626544a4ac8232'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:44.061766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-191f9780-79a0-4753-a10c-f24958d2c0a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e68c038d010f472bb0626544a4ac8232'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:44.063055 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-191f9780-79a0-4753-a10c-f24958d2c0a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e68c038d010f472bb0626544a4ac8232'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:44.071359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-191f9780-79a0-4753-a10c-f24958d2c0a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e68c038d010f472bb0626544a4ac8232'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:44.090828 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1180/2340] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:45:43 2022] DELETE /identity/v3/users/e68c038d010f472bb0626544a4ac8232 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:44.549259 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:44.550842 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:44.569185 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f62b7-f9eb-4e50-a538-c003b5a88ce0 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:44.634143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fce89-b26d-467e-a3d3-36eab4678d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:44.634778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fce89-b26d-467e-a3d3-36eab4678d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:44.635048 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fce89-b26d-467e-a3d3-36eab4678d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:44.635318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fce89-b26d-467e-a3d3-36eab4678d58 admin admin] PATH_INFO: `/v3/projects/ec76534c9c244880b7710263c3cff857` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:44.636872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fce89-b26d-467e-a3d3-36eab4678d58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec76534c9c244880b7710263c3cff857)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:44.638932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fce89-b26d-467e-a3d3-36eab4678d58 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:44.741097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8a6fce89-b26d-467e-a3d3-36eab4678d58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec76534c9c244880b7710263c3cff857', 'request_id': 'req-8a6fce89-b26d-467e-a3d3-36eab4678d58'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:44.747911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8a6fce89-b26d-467e-a3d3-36eab4678d58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec76534c9c244880b7710263c3cff857', 'request_id': 'req-8a6fce89-b26d-467e-a3d3-36eab4678d58'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:44.755026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8a6fce89-b26d-467e-a3d3-36eab4678d58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec76534c9c244880b7710263c3cff857', 'request_id': 'req-8a6fce89-b26d-467e-a3d3-36eab4678d58'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:44.810370 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8a6fce89-b26d-467e-a3d3-36eab4678d58 admin admin] The token cache is being invalidate because project ec76534c9c244880b7710263c3cff857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:45:44.816036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8a6fce89-b26d-467e-a3d3-36eab4678d58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:44.816036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8a6fce89-b26d-467e-a3d3-36eab4678d58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:44.816036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1161/2341] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:45:44 2022] DELETE /identity/v3/projects/ec76534c9c244880b7710263c3cff857 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:44.819627 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b95957-08e5-479f-90d1-8ca237200741 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:44.820111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b95957-08e5-479f-90d1-8ca237200741 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:44.820228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b95957-08e5-479f-90d1-8ca237200741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:44.839849 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c6b95957-08e5-479f-90d1-8ca237200741 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:44.844961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c6b95957-08e5-479f-90d1-8ca237200741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:44.900050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1181/2342] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:44.908149 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4718f-9d44-442e-b2c4-17d2bfb975dd None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:44.908434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4718f-9d44-442e-b2c4-17d2bfb975dd None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:44.908755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4718f-9d44-442e-b2c4-17d2bfb975dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:44.930445 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-14c4718f-9d44-442e-b2c4-17d2bfb975dd None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:44.931648 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-14c4718f-9d44-442e-b2c4-17d2bfb975dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:44.939539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1162/2343] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:44.943964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b95957-08e5-479f-90d1-8ca237200741 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:44.953596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b95957-08e5-479f-90d1-8ca237200741 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:44.958686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09ed4a-da5d-4137-9220-200a4af07c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:44.960173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09ed4a-da5d-4137-9220-200a4af07c2f admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:44.960173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09ed4a-da5d-4137-9220-200a4af07c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:44.960173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09ed4a-da5d-4137-9220-200a4af07c2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:44.960447 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09ed4a-da5d-4137-9220-200a4af07c2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:45:44.960705 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09ed4a-da5d-4137-9220-200a4af07c2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:44.962312 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09ed4a-da5d-4137-9220-200a4af07c2f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:44.973793 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1182/2344] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:45:44.978429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4718f-9d44-442e-b2c4-17d2bfb975dd None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:44.987760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4718f-9d44-442e-b2c4-17d2bfb975dd None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:44.993967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae3f85-522d-4416-a863-0211a57e53ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:44.994556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae3f85-522d-4416-a863-0211a57e53ed admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:44.994828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae3f85-522d-4416-a863-0211a57e53ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:44.995089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae3f85-522d-4416-a863-0211a57e53ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:44.995717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae3f85-522d-4416-a863-0211a57e53ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:45.003229 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae3f85-522d-4416-a863-0211a57e53ed admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:45.011373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:45.105529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1163/2345] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:45:44 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:45.109184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09ed4a-da5d-4137-9220-200a4af07c2f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:45.119983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09ed4a-da5d-4137-9220-200a4af07c2f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:45.134072 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:45.179529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975834fe-1237-4f85-bc65-4acd58926ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:45.180175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975834fe-1237-4f85-bc65-4acd58926ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:45.180456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975834fe-1237-4f85-bc65-4acd58926ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:45.180735 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975834fe-1237-4f85-bc65-4acd58926ef9 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:45.181403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975834fe-1237-4f85-bc65-4acd58926ef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:45.183339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975834fe-1237-4f85-bc65-4acd58926ef9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:45.279404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1183/2346] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:45:45 2022] POST /identity/v3/users => generated 291 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:45.283554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae3f85-522d-4416-a863-0211a57e53ed admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:45.295163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae3f85-522d-4416-a863-0211a57e53ed admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:45.299714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c3435-bd6b-4efe-ac37-e8a3dc01fdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:45.305451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c3435-bd6b-4efe-ac37-e8a3dc01fdac admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:45.305451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c3435-bd6b-4efe-ac37-e8a3dc01fdac admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:45.305451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c3435-bd6b-4efe-ac37-e8a3dc01fdac admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:45.305451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c3435-bd6b-4efe-ac37-e8a3dc01fdac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:45.305451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c3435-bd6b-4efe-ac37-e8a3dc01fdac admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:45.314258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1164/2347] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:45.318797 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975834fe-1237-4f85-bc65-4acd58926ef9 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:45.328885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975834fe-1237-4f85-bc65-4acd58926ef9 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:45.333575 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112bcde8-8d14-443b-8e9f-a1f89b752ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:45.336120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bcde8-8d14-443b-8e9f-a1f89b752ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:45.336120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bcde8-8d14-443b-8e9f-a1f89b752ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:45.336120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112bcde8-8d14-443b-8e9f-a1f89b752ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:45.336120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112bcde8-8d14-443b-8e9f-a1f89b752ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:45.337045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112bcde8-8d14-443b-8e9f-a1f89b752ff8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:45.349459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1184/2348] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:45.354069 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c3435-bd6b-4efe-ac37-e8a3dc01fdac admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:45.364857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c3435-bd6b-4efe-ac37-e8a3dc01fdac admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:45.369208 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407ec3d-3876-4706-9551-25d063cef631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:45.369873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407ec3d-3876-4706-9551-25d063cef631 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:45.370148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407ec3d-3876-4706-9551-25d063cef631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:45.370418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407ec3d-3876-4706-9551-25d063cef631 admin admin] PATH_INFO: `/v3/projects/80fb28c07409469599fc0d7418098ddc/users/2bb485dc285548c08db19563b3c29cd4/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:45.384850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407ec3d-3876-4706-9551-25d063cef631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80fb28c07409469599fc0d7418098ddc, user_id=2bb485dc285548c08db19563b3c29cd4, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:45.387551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407ec3d-3876-4706-9551-25d063cef631 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:45.455733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1165/2349] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:45:45 2022] PUT /identity/v3/projects/80fb28c07409469599fc0d7418098ddc/users/2bb485dc285548c08db19563b3c29cd4/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:45.477463 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137c543-4cea-4e6a-8946-8defa9699374 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:45.477748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137c543-4cea-4e6a-8946-8defa9699374 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:45.478129 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137c543-4cea-4e6a-8946-8defa9699374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:45.505721 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-d137c543-4cea-4e6a-8946-8defa9699374 None None] MFA Rules not processed for user `2bb485dc285548c08db19563b3c29cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:45.506955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d137c543-4cea-4e6a-8946-8defa9699374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:45.571789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1185/2350] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:45.576380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407ec3d-3876-4706-9551-25d063cef631 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:45.641683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407ec3d-3876-4706-9551-25d063cef631 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:45.647657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f91424-ba00-46e1-84a1-a46c774e7185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:45.648264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f91424-ba00-46e1-84a1-a46c774e7185 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:45.648537 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f91424-ba00-46e1-84a1-a46c774e7185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:45.648800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f91424-ba00-46e1-84a1-a46c774e7185 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:45.649429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f91424-ba00-46e1-84a1-a46c774e7185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:45.651344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f91424-ba00-46e1-84a1-a46c774e7185 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:45.770974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1166/2351] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:45:45 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:45.775101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137c543-4cea-4e6a-8946-8defa9699374 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:45.786629 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137c543-4cea-4e6a-8946-8defa9699374 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:45.868599 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75ffcb-6f71-4cb1-8b89-7509ea683ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:45.869200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75ffcb-6f71-4cb1-8b89-7509ea683ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:45.869568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75ffcb-6f71-4cb1-8b89-7509ea683ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:45.869864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75ffcb-6f71-4cb1-8b89-7509ea683ef4 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:45.870501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75ffcb-6f71-4cb1-8b89-7509ea683ef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:45.872391 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75ffcb-6f71-4cb1-8b89-7509ea683ef4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:45.960084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1186/2352] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:45:45 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:45.965748 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f91424-ba00-46e1-84a1-a46c774e7185 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:45.977206 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f91424-ba00-46e1-84a1-a46c774e7185 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:45.982021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d121a-e8b6-4094-b6a6-f33288f18c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:45.982644 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d121a-e8b6-4094-b6a6-f33288f18c2a admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:45.982940 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d121a-e8b6-4094-b6a6-f33288f18c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:45.983231 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d121a-e8b6-4094-b6a6-f33288f18c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:45.983862 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d121a-e8b6-4094-b6a6-f33288f18c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:45.985425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d121a-e8b6-4094-b6a6-f33288f18c2a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:45.996608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1167/2353] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:46.000833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75ffcb-6f71-4cb1-8b89-7509ea683ef4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:46.011329 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75ffcb-6f71-4cb1-8b89-7509ea683ef4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:46.016184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cfcf9-d9ff-4fc9-8c62-17cb5ee9ad5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:46.016779 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cfcf9-d9ff-4fc9-8c62-17cb5ee9ad5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:46.017050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cfcf9-d9ff-4fc9-8c62-17cb5ee9ad5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:46.017317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cfcf9-d9ff-4fc9-8c62-17cb5ee9ad5b admin admin] PATH_INFO: `/v3/projects/ebe7c56c8628445f88d634edfb06b5d6/users/acd0f5d8f4a34567899535261fe08052/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:46.030590 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:46.035226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cfcf9-d9ff-4fc9-8c62-17cb5ee9ad5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebe7c56c8628445f88d634edfb06b5d6, user_id=acd0f5d8f4a34567899535261fe08052, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:46.038536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cfcf9-d9ff-4fc9-8c62-17cb5ee9ad5b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:46.145757 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1187/2354] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:45:45 2022] PUT /identity/v3/projects/ebe7c56c8628445f88d634edfb06b5d6/users/acd0f5d8f4a34567899535261fe08052/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:46.149964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d121a-e8b6-4094-b6a6-f33288f18c2a admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:46.165515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d121a-e8b6-4094-b6a6-f33288f18c2a admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:46.239760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce377a-b2d5-4c51-8920-4d28ad7dc76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:46.240835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce377a-b2d5-4c51-8920-4d28ad7dc76c admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:46.241202 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce377a-b2d5-4c51-8920-4d28ad7dc76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:46.241565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce377a-b2d5-4c51-8920-4d28ad7dc76c admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:46.242348 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce377a-b2d5-4c51-8920-4d28ad7dc76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:46.244084 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce377a-b2d5-4c51-8920-4d28ad7dc76c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:46.257870 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1168/2355] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:46.260294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cfcf9-d9ff-4fc9-8c62-17cb5ee9ad5b admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:46.272073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cfcf9-d9ff-4fc9-8c62-17cb5ee9ad5b admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:46.276789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a931187-4ed7-49b1-aca8-c4a297becd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:46.277495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a931187-4ed7-49b1-aca8-c4a297becd5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:46.277772 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a931187-4ed7-49b1-aca8-c4a297becd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:46.278067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a931187-4ed7-49b1-aca8-c4a297becd5b admin admin] PATH_INFO: `/v3/projects/ebe7c56c8628445f88d634edfb06b5d6/users/acd0f5d8f4a34567899535261fe08052/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:46.280638 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a931187-4ed7-49b1-aca8-c4a297becd5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebe7c56c8628445f88d634edfb06b5d6, user_id=acd0f5d8f4a34567899535261fe08052, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:46.283363 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a931187-4ed7-49b1-aca8-c4a297becd5b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:46.305924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1188/2356] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:45:46 2022] PUT /identity/v3/projects/ebe7c56c8628445f88d634edfb06b5d6/users/acd0f5d8f4a34567899535261fe08052/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:46.309200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f82db1-dabb-4f42-bfe8-26f31435c43b None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:46.309483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f82db1-dabb-4f42-bfe8-26f31435c43b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:46.309752 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f82db1-dabb-4f42-bfe8-26f31435c43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:46.342937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-d8f82db1-dabb-4f42-bfe8-26f31435c43b None None] MFA Rules not processed for user `acd0f5d8f4a34567899535261fe08052`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:46.344263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d8f82db1-dabb-4f42-bfe8-26f31435c43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:46.359335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:46.424544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1169/2357] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:46 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:46.437019 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a931187-4ed7-49b1-aca8-c4a297becd5b admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:46.438862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2a931187-4ed7-49b1-aca8-c4a297becd5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:46.505919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a931187-4ed7-49b1-aca8-c4a297becd5b admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:46.509918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545e52e-6915-4af5-906c-8087727fc3bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:46.510540 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545e52e-6915-4af5-906c-8087727fc3bb service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:46.510817 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545e52e-6915-4af5-906c-8087727fc3bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:46.511086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545e52e-6915-4af5-906c-8087727fc3bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:46.521634 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545e52e-6915-4af5-906c-8087727fc3bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:46.523122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545e52e-6915-4af5-906c-8087727fc3bb service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:46.533581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1189/2358] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:46 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:45:46.718091 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f82db1-dabb-4f42-bfe8-26f31435c43b None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:46.732444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f82db1-dabb-4f42-bfe8-26f31435c43b None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:46.736819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f493a-e18f-4125-b825-b68268cc32a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:46.737434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f493a-e18f-4125-b825-b68268cc32a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:46.738039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f493a-e18f-4125-b825-b68268cc32a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:46.738039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f493a-e18f-4125-b825-b68268cc32a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:46.845778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f493a-e18f-4125-b825-b68268cc32a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:46.847875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f493a-e18f-4125-b825-b68268cc32a2 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:46.873476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1170/2359] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:47.460782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545e52e-6915-4af5-906c-8087727fc3bb service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:47.523364 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545e52e-6915-4af5-906c-8087727fc3bb service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:47.528108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:47.528987 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:47.529255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:47.529518 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] PATH_INFO: `/v3/users/2bb485dc285548c08db19563b3c29cd4` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:47.530640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bb485dc285548c08db19563b3c29cd4)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:47.532622 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:47.681339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bb485dc285548c08db19563b3c29cd4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:47.682021 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bb485dc285548c08db19563b3c29cd4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:47.747645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bb485dc285548c08db19563b3c29cd4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:47.748245 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bb485dc285548c08db19563b3c29cd4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:47.755654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bb485dc285548c08db19563b3c29cd4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:47.770184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:47.774645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1190/2360] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:45:47 2022] DELETE /identity/v3/users/2bb485dc285548c08db19563b3c29cd4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:45:47.779471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9043cc-6731-4496-9786-93d8555bd6ba None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:47.779746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9043cc-6731-4496-9786-93d8555bd6ba None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:47.780009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9043cc-6731-4496-9786-93d8555bd6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:47.799249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-8e9043cc-6731-4496-9786-93d8555bd6ba None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:47.800499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8e9043cc-6731-4496-9786-93d8555bd6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:47.859465 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1171/2361] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:47.869484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:47.870895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:47.935230 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074dd3f-0b05-4549-88bd-61a70ea0a4c3 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:47.940280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39612b12-452a-4b30-b859-46293cb5a20e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:47.941040 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39612b12-452a-4b30-b859-46293cb5a20e service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:47.941415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39612b12-452a-4b30-b859-46293cb5a20e service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:47.941779 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39612b12-452a-4b30-b859-46293cb5a20e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:47.967417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39612b12-452a-4b30-b859-46293cb5a20e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:47.967417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39612b12-452a-4b30-b859-46293cb5a20e service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:47.971724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1191/2362] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:48.670783 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9043cc-6731-4496-9786-93d8555bd6ba None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:48.675098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8e9043cc-6731-4496-9786-93d8555bd6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:48.685522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9043cc-6731-4496-9786-93d8555bd6ba None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:48.690192 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:48.690837 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:48.691078 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:48.691339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] PATH_INFO: `/v3/projects/80fb28c07409469599fc0d7418098ddc` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:48.692436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80fb28c07409469599fc0d7418098ddc)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:48.694384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:48.812608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:48.816371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80fb28c07409469599fc0d7418098ddc', 'request_id': 'req-c531abfa-b195-4898-80eb-014394c73b38'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:48.822420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80fb28c07409469599fc0d7418098ddc', 'request_id': 'req-c531abfa-b195-4898-80eb-014394c73b38'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:48.829229 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80fb28c07409469599fc0d7418098ddc', 'request_id': 'req-c531abfa-b195-4898-80eb-014394c73b38'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:48.965248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] The token cache is being invalidate because project 80fb28c07409469599fc0d7418098ddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:45:48.966009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:48.966217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:48.968501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1172/2363] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:45:48 2022] DELETE /identity/v3/projects/80fb28c07409469599fc0d7418098ddc => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:48.971800 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39612b12-452a-4b30-b859-46293cb5a20e service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:48.981533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39612b12-452a-4b30-b859-46293cb5a20e service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:49.038071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:49.038887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:49.039314 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:49.039590 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] PATH_INFO: `/v3/users/acd0f5d8f4a34567899535261fe08052` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:49.040709 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acd0f5d8f4a34567899535261fe08052)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:49.042645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:49.225232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acd0f5d8f4a34567899535261fe08052'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:49.226295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acd0f5d8f4a34567899535261fe08052'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:49.296174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acd0f5d8f4a34567899535261fe08052'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:49.296821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acd0f5d8f4a34567899535261fe08052'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:49.305540 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acd0f5d8f4a34567899535261fe08052'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:49.313552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:49.315588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:49.338040 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1192/2364] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:45:48 2022] DELETE /identity/v3/users/acd0f5d8f4a34567899535261fe08052 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:49.415045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531abfa-b195-4898-80eb-014394c73b38 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:49.488854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718585fd-e8e6-4063-96bd-5a316640ef4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9975c0dab76f4ccbb4d2107e8e780ae5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:49.489462 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718585fd-e8e6-4063-96bd-5a316640ef4a service placement] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:49.489733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718585fd-e8e6-4063-96bd-5a316640ef4a service placement] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:49.490025 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718585fd-e8e6-4063-96bd-5a316640ef4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:49.492486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-718585fd-e8e6-4063-96bd-5a316640ef4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:49.506212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718585fd-e8e6-4063-96bd-5a316640ef4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:49.506212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718585fd-e8e6-4063-96bd-5a316640ef4a service placement] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:49.516751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1173/2365] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:45:49.746123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:49.747325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:49.776441 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:49.815456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee9d6c4-78af-4d7f-a571-26a6a6d670c4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:49.820555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:49.821173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:49.821444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:49.821706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c admin admin] PATH_INFO: `/v3/projects/ebe7c56c8628445f88d634edfb06b5d6` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:49.822839 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebe7c56c8628445f88d634edfb06b5d6)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:49.824719 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:49.920032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebe7c56c8628445f88d634edfb06b5d6', 'request_id': 'req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:49.925419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebe7c56c8628445f88d634edfb06b5d6', 'request_id': 'req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:49.932083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebe7c56c8628445f88d634edfb06b5d6', 'request_id': 'req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:50.003886 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c admin admin] The token cache is being invalidate because project ebe7c56c8628445f88d634edfb06b5d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:45:50.004464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:50.004981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6d42f6d1-61be-4815-b5ed-9b96eca7b45c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:45:50.006495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1193/2366] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:45:49 2022] DELETE /identity/v3/projects/ebe7c56c8628445f88d634edfb06b5d6 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:50.012379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f58b6-0918-499b-81c8-cd15a5b3fb67 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:50.013451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f58b6-0918-499b-81c8-cd15a5b3fb67 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:50.013730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f58b6-0918-499b-81c8-cd15a5b3fb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:50.033599 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-023f58b6-0918-499b-81c8-cd15a5b3fb67 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:50.034819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-023f58b6-0918-499b-81c8-cd15a5b3fb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:50.099801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1174/2367] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:50.110163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9158526-f451-4a4c-848a-32c5de098e1b None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:50.110750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9158526-f451-4a4c-848a-32c5de098e1b None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:50.110750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9158526-f451-4a4c-848a-32c5de098e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:50.131872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c9158526-f451-4a4c-848a-32c5de098e1b None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:50.133708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c9158526-f451-4a4c-848a-32c5de098e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:50.142287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1194/2368] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:50.146703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f58b6-0918-499b-81c8-cd15a5b3fb67 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:50.152739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:50.156837 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f58b6-0918-499b-81c8-cd15a5b3fb67 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:50.161616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1828f-c5b1-4a22-84b9-d08e478d8ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:50.166065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1828f-c5b1-4a22-84b9-d08e478d8ccc admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:50.166065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1828f-c5b1-4a22-84b9-d08e478d8ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:50.166065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1828f-c5b1-4a22-84b9-d08e478d8ccc admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:50.166065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1828f-c5b1-4a22-84b9-d08e478d8ccc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:45:50.166065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1828f-c5b1-4a22-84b9-d08e478d8ccc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:50.166065 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1828f-c5b1-4a22-84b9-d08e478d8ccc admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:50.176737 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1175/2369] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:45:50.181694 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9158526-f451-4a4c-848a-32c5de098e1b None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:50.190929 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9158526-f451-4a4c-848a-32c5de098e1b None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:50.196271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73733903-5678-4a4a-be9f-d7895ef9e097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:50.196869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73733903-5678-4a4a-be9f-d7895ef9e097 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:50.197143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73733903-5678-4a4a-be9f-d7895ef9e097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:50.197523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73733903-5678-4a4a-be9f-d7895ef9e097 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:50.198187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73733903-5678-4a4a-be9f-d7895ef9e097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:50.200072 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73733903-5678-4a4a-be9f-d7895ef9e097 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:50.303508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1195/2370] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:45:50 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:50.308074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1828f-c5b1-4a22-84b9-d08e478d8ccc admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:50.318520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1828f-c5b1-4a22-84b9-d08e478d8ccc admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:50.374372 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3c91a-1d58-454c-a5c7-2ff763383112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:50.375009 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3c91a-1d58-454c-a5c7-2ff763383112 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:50.375260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3c91a-1d58-454c-a5c7-2ff763383112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:50.375525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3c91a-1d58-454c-a5c7-2ff763383112 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:50.376139 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3c91a-1d58-454c-a5c7-2ff763383112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:50.378042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3c91a-1d58-454c-a5c7-2ff763383112 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:50.457022 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1176/2371] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:45:50 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:45:50.462574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73733903-5678-4a4a-be9f-d7895ef9e097 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:50.472944 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73733903-5678-4a4a-be9f-d7895ef9e097 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:50.479490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6704f7e-6c91-4da7-a46b-e755d29d8bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:50.480094 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6704f7e-6c91-4da7-a46b-e755d29d8bfe admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:50.480366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6704f7e-6c91-4da7-a46b-e755d29d8bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:50.480629 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6704f7e-6c91-4da7-a46b-e755d29d8bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:50.481266 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6704f7e-6c91-4da7-a46b-e755d29d8bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:50.482925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6704f7e-6c91-4da7-a46b-e755d29d8bfe admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:50.494193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1196/2372] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:45:50.508523 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3c91a-1d58-454c-a5c7-2ff763383112 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:50.518483 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3c91a-1d58-454c-a5c7-2ff763383112 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:50.523651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110cc6f3-551e-4eed-b7de-de4ff306b0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:50.524281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cc6f3-551e-4eed-b7de-de4ff306b0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:50.524551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cc6f3-551e-4eed-b7de-de4ff306b0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:50.524815 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cc6f3-551e-4eed-b7de-de4ff306b0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:50.525446 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110cc6f3-551e-4eed-b7de-de4ff306b0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:50.527484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110cc6f3-551e-4eed-b7de-de4ff306b0d7 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:50.540057 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1177/2373] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:45:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:45:50.544148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6704f7e-6c91-4da7-a46b-e755d29d8bfe admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:50.555390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6704f7e-6c91-4da7-a46b-e755d29d8bfe admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:50.560158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f82ae2-45da-4bb6-92b6-80605fb2d130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:50.560798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f82ae2-45da-4bb6-92b6-80605fb2d130 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:50.561073 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f82ae2-45da-4bb6-92b6-80605fb2d130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:50.561341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f82ae2-45da-4bb6-92b6-80605fb2d130 admin admin] PATH_INFO: `/v3/projects/3bdb629ccf81494ebcc1192615e1187e/users/dc1aba6d01fe441aa740af05bc66074e/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:50.576497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f82ae2-45da-4bb6-92b6-80605fb2d130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bdb629ccf81494ebcc1192615e1187e, user_id=dc1aba6d01fe441aa740af05bc66074e, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:50.579183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f82ae2-45da-4bb6-92b6-80605fb2d130 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:50.661770 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1197/2374] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:45:50 2022] PUT /identity/v3/projects/3bdb629ccf81494ebcc1192615e1187e/users/dc1aba6d01fe441aa740af05bc66074e/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:45:50.728472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a0f11-960a-47ac-98bb-6d60be655291 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:50.728752 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a0f11-960a-47ac-98bb-6d60be655291 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:50.729024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a0f11-960a-47ac-98bb-6d60be655291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:50.760334 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-529a0f11-960a-47ac-98bb-6d60be655291 None None] MFA Rules not processed for user `dc1aba6d01fe441aa740af05bc66074e`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:50.761811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-529a0f11-960a-47ac-98bb-6d60be655291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:50.829448 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1178/2375] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:45:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:50.840449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f82ae2-45da-4bb6-92b6-80605fb2d130 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:50.842155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-87f82ae2-45da-4bb6-92b6-80605fb2d130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:50.895160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:50.898754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f82ae2-45da-4bb6-92b6-80605fb2d130 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:50.906369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e114e89-f50b-4f44-9dd9-6df649011c1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:50.906369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e114e89-f50b-4f44-9dd9-6df649011c1d service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:50.906369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e114e89-f50b-4f44-9dd9-6df649011c1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:50.906369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e114e89-f50b-4f44-9dd9-6df649011c1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:50.912417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e114e89-f50b-4f44-9dd9-6df649011c1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:50.913973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e114e89-f50b-4f44-9dd9-6df649011c1d service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:50.925095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1198/2376] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:53.265402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc358cd3-ddf9-4624-97d7-2cee24ea553f None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:53.265959 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc358cd3-ddf9-4624-97d7-2cee24ea553f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:53.266090 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc358cd3-ddf9-4624-97d7-2cee24ea553f None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:53.267680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1179/2377] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:45:53.273857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4ce77-2eba-4626-92cc-1adebcaca00e None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:53.274335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4ce77-2eba-4626-92cc-1adebcaca00e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:53.274452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4ce77-2eba-4626-92cc-1adebcaca00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:53.295857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-4dd4ce77-2eba-4626-92cc-1adebcaca00e None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:53.297057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4dd4ce77-2eba-4626-92cc-1adebcaca00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:53.368808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1199/2378] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:45:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:53.380903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc358cd3-ddf9-4624-97d7-2cee24ea553f None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:53.384089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-fc358cd3-ddf9-4624-97d7-2cee24ea553f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:53.443694 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc358cd3-ddf9-4624-97d7-2cee24ea553f None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:53.447766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83429826-b1ac-45ed-8309-5fc778920196 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d96ec6235843c5b67d87a3d0ff0f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:53.448370 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83429826-b1ac-45ed-8309-5fc778920196 service glance] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:53.448646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83429826-b1ac-45ed-8309-5fc778920196 service glance] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:53.448914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83429826-b1ac-45ed-8309-5fc778920196 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:53.459339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83429826-b1ac-45ed-8309-5fc778920196 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:53.460752 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83429826-b1ac-45ed-8309-5fc778920196 service glance] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:53.471820 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1180/2379] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:45:53.663446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1064d08-5803-4330-b1bc-670845f6ddc1 None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:53.664482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1064d08-5803-4330-b1bc-670845f6ddc1 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:53.664655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1064d08-5803-4330-b1bc-670845f6ddc1 None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:53.666047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1200/2380] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:45:53.670809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db33479-2b8a-471b-b4f1-317e01bc316f None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:53.671094 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db33479-2b8a-471b-b4f1-317e01bc316f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:53.671359 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db33479-2b8a-471b-b4f1-317e01bc316f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:53.679034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:45:53.691840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-1db33479-2b8a-471b-b4f1-317e01bc316f None None] MFA Rules not processed for user `3e923be9aeed43728690d24d85dbe062`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:53.693034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1db33479-2b8a-471b-b4f1-317e01bc316f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:53.746342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1181/2381] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:45:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:53.759303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1064d08-5803-4330-b1bc-670845f6ddc1 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:53.761004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a1064d08-5803-4330-b1bc-670845f6ddc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:53.774987 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1064d08-5803-4330-b1bc-670845f6ddc1 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:53.778750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a27ca28-a513-4ee4-aa56-c434893cd437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:53.779365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ca28-a513-4ee4-aa56-c434893cd437 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:53.779620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ca28-a513-4ee4-aa56-c434893cd437 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:53.779888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27ca28-a513-4ee4-aa56-c434893cd437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:53.786698 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27ca28-a513-4ee4-aa56-c434893cd437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:53.788109 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a27ca28-a513-4ee4-aa56-c434893cd437 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:53.796113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1201/2382] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:53.877438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251317b5-2856-4f9b-95d5-685bb79c2a7f None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:53.877742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251317b5-2856-4f9b-95d5-685bb79c2a7f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:53.878040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251317b5-2856-4f9b-95d5-685bb79c2a7f None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:53.879365 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1182/2383] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:45:53.885654 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796377d9-5f7c-4184-9f6e-bf7ea510da79 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:53.885994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796377d9-5f7c-4184-9f6e-bf7ea510da79 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:53.886269 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796377d9-5f7c-4184-9f6e-bf7ea510da79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:53.911973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-796377d9-5f7c-4184-9f6e-bf7ea510da79 None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:45:53.913365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-796377d9-5f7c-4184-9f6e-bf7ea510da79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:45:53.924300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1202/2384] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:45:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:45:53.935454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251317b5-2856-4f9b-95d5-685bb79c2a7f None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:53.946700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251317b5-2856-4f9b-95d5-685bb79c2a7f None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:45:53.950218 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819d703-3343-4975-954b-c18327737137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:45:53.950808 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819d703-3343-4975-954b-c18327737137 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:45:53.951074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819d703-3343-4975-954b-c18327737137 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:45:53.951382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819d703-3343-4975-954b-c18327737137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:45:53.961618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819d703-3343-4975-954b-c18327737137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:45:53.963326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819d703-3343-4975-954b-c18327737137 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:45:53.974194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1183/2385] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:45:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:45:59.991364 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796377d9-5f7c-4184-9f6e-bf7ea510da79 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:45:59.993789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-796377d9-5f7c-4184-9f6e-bf7ea510da79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:00.072203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796377d9-5f7c-4184-9f6e-bf7ea510da79 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:00.076760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:00.077358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:00.077700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:00.078055 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] PATH_INFO: `/v3/users/3ab4610009394582a3ebdb97164e6054` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:00.079413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab4610009394582a3ebdb97164e6054)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:00.081460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:00.282744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab4610009394582a3ebdb97164e6054'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:00.283050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab4610009394582a3ebdb97164e6054'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:00.376492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab4610009394582a3ebdb97164e6054'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:00.377138 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab4610009394582a3ebdb97164e6054'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:00.385216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab4610009394582a3ebdb97164e6054'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:00.387450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:00.404191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1203/2386] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:45:59 2022] DELETE /identity/v3/users/3ab4610009394582a3ebdb97164e6054 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:46:00.415049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b76db-7a90-423a-a524-953d8a37d00b None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:00.415049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b76db-7a90-423a-a524-953d8a37d00b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:00.415049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b76db-7a90-423a-a524-953d8a37d00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:00.429906 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-830b76db-7a90-423a-a524-953d8a37d00b None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:00.431183 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-830b76db-7a90-423a-a524-953d8a37d00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:00.490613 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1184/2387] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:46:00.501247 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:00.502660 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:00.560320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3e30e-4b81-4a61-bc24-6828b7ad6619 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:00.563834 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ad3b8-6000-4798-85c0-a02bc6e93a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:00.564435 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ad3b8-6000-4798-85c0-a02bc6e93a86 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:00.564711 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ad3b8-6000-4798-85c0-a02bc6e93a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:00.564979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ad3b8-6000-4798-85c0-a02bc6e93a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:00.574766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ad3b8-6000-4798-85c0-a02bc6e93a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:00.576180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ad3b8-6000-4798-85c0-a02bc6e93a86 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:00.585563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1204/2388] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:46:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:46:01.017799 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b76db-7a90-423a-a524-953d8a37d00b None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:01.019263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-830b76db-7a90-423a-a524-953d8a37d00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:01.032708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b76db-7a90-423a-a524-953d8a37d00b None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:01.037335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:01.038023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:01.038351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:01.038654 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] PATH_INFO: `/v3/projects/1c3b04ba9b2844508592bcf1aefd7859` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:01.040126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c3b04ba9b2844508592bcf1aefd7859)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:01.041780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:01.124595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c3b04ba9b2844508592bcf1aefd7859', 'request_id': 'req-4da27d8e-f801-45ce-9275-585c002f9cfc'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:01.130341 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c3b04ba9b2844508592bcf1aefd7859', 'request_id': 'req-4da27d8e-f801-45ce-9275-585c002f9cfc'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:01.137666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c3b04ba9b2844508592bcf1aefd7859', 'request_id': 'req-4da27d8e-f801-45ce-9275-585c002f9cfc'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:01.214555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] The token cache is being invalidate because project 1c3b04ba9b2844508592bcf1aefd7859 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:46:01.214791 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:01.215384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:01.216850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1185/2389] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:46:01 2022] DELETE /identity/v3/projects/1c3b04ba9b2844508592bcf1aefd7859 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:46:10.581205 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=32,keystone:DELETE=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:11.211266 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:13.722761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00a5f2-57bd-4bfa-ad8b-292413f3532c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:13.723290 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00a5f2-57bd-4bfa-ad8b-292413f3532c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:13.723402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00a5f2-57bd-4bfa-ad8b-292413f3532c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:13.745030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-7f00a5f2-57bd-4bfa-ad8b-292413f3532c None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:13.746348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7f00a5f2-57bd-4bfa-ad8b-292413f3532c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:13.801764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1205/2390] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:46:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:46:13.813796 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:13.815217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:13.889842 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da27d8e-f801-45ce-9275-585c002f9cfc admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:13.895074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5850c9-26b8-48e4-90ea-5f0c0259793d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:13.895749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5850c9-26b8-48e4-90ea-5f0c0259793d service swift] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:13.896031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5850c9-26b8-48e4-90ea-5f0c0259793d service swift] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:13.896333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5850c9-26b8-48e4-90ea-5f0c0259793d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:13.905116 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5850c9-26b8-48e4-90ea-5f0c0259793d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:13.906801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5850c9-26b8-48e4-90ea-5f0c0259793d service swift] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:13.915337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1186/2391] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:46:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 16 05:46:23.795488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=25 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:23.911822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=32 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:27.962818 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00a5f2-57bd-4bfa-ad8b-292413f3532c None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:27.968994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7f00a5f2-57bd-4bfa-ad8b-292413f3532c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:28.032333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00a5f2-57bd-4bfa-ad8b-292413f3532c None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:28.036857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:28.037474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:28.037752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:28.038149 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] PATH_INFO: `/v3/users/85db699157c1461e92cc42fe3a86a150` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:28.039422 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=85db699157c1461e92cc42fe3a86a150)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:28.043049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:28.253502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85db699157c1461e92cc42fe3a86a150'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:28.254163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85db699157c1461e92cc42fe3a86a150'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:28.324981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85db699157c1461e92cc42fe3a86a150'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:28.325589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85db699157c1461e92cc42fe3a86a150'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:28.333720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85db699157c1461e92cc42fe3a86a150'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:28.352315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1206/2392] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:46:27 2022] DELETE /identity/v3/users/85db699157c1461e92cc42fe3a86a150 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:46:28.357142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f686b62-5b86-4539-9522-df6807b02d04 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:28.364352 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f686b62-5b86-4539-9522-df6807b02d04 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:28.364352 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f686b62-5b86-4539-9522-df6807b02d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:28.376501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-9f686b62-5b86-4539-9522-df6807b02d04 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:28.377680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9f686b62-5b86-4539-9522-df6807b02d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:28.441463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1187/2393] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:46:28.451394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:28.452990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:28.509901 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fc0dc-2060-4a75-af2b-f865ce795cae admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:28.513468 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30931ddb-8ad9-43f8-86bd-56fae3df3f6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:28.514123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30931ddb-8ad9-43f8-86bd-56fae3df3f6c service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:28.514396 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30931ddb-8ad9-43f8-86bd-56fae3df3f6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:28.514662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30931ddb-8ad9-43f8-86bd-56fae3df3f6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:28.524471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30931ddb-8ad9-43f8-86bd-56fae3df3f6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:28.525973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30931ddb-8ad9-43f8-86bd-56fae3df3f6c service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:28.536625 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1207/2394] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:46:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:46:28.986987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f686b62-5b86-4539-9522-df6807b02d04 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:28.988356 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9f686b62-5b86-4539-9522-df6807b02d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:29.004340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f686b62-5b86-4539-9522-df6807b02d04 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:29.009393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:29.010014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:29.010293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:29.010565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc admin admin] PATH_INFO: `/v3/projects/ce5edc1d496a4318aba96b0df7b3ba68` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:29.011769 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce5edc1d496a4318aba96b0df7b3ba68)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:29.013671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:29.130024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce5edc1d496a4318aba96b0df7b3ba68', 'request_id': 'req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:29.135890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce5edc1d496a4318aba96b0df7b3ba68', 'request_id': 'req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:29.146285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce5edc1d496a4318aba96b0df7b3ba68', 'request_id': 'req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:29.229857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc admin admin] The token cache is being invalidate because project ce5edc1d496a4318aba96b0df7b3ba68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:46:29.230136 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:29.230800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3f5dc315-5f2e-42fc-87d1-5ad1bfc236bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:29.232262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1188/2395] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:46:28 2022] DELETE /identity/v3/projects/ce5edc1d496a4318aba96b0df7b3ba68 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:46:29.238181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a950c0c-1e88-4bc3-a924-db7258a7c0c6 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:29.238665 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a950c0c-1e88-4bc3-a924-db7258a7c0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:29.238781 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a950c0c-1e88-4bc3-a924-db7258a7c0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:29.258862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-7a950c0c-1e88-4bc3-a924-db7258a7c0c6 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:29.260056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7a950c0c-1e88-4bc3-a924-db7258a7c0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:29.276926 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:29.319245 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1208/2396] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:46:29.393527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328d9e3-f1d8-47ae-b140-070526753f04 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:29.394125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328d9e3-f1d8-47ae-b140-070526753f04 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:29.394125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328d9e3-f1d8-47ae-b140-070526753f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:29.414038 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-6328d9e3-f1d8-47ae-b140-070526753f04 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:29.415265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6328d9e3-f1d8-47ae-b140-070526753f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:29.423699 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1189/2397] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:46:29.431512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a950c0c-1e88-4bc3-a924-db7258a7c0c6 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:29.443231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a950c0c-1e88-4bc3-a924-db7258a7c0c6 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:29.448185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891e4b9-1e41-48c9-8c3d-d52b8f3511b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:29.448784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891e4b9-1e41-48c9-8c3d-d52b8f3511b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:29.449067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891e4b9-1e41-48c9-8c3d-d52b8f3511b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:29.449335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d891e4b9-1e41-48c9-8c3d-d52b8f3511b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:29.450085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891e4b9-1e41-48c9-8c3d-d52b8f3511b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:46:29.450382 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891e4b9-1e41-48c9-8c3d-d52b8f3511b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:29.452300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d891e4b9-1e41-48c9-8c3d-d52b8f3511b3 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:29.464909 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1209/2398] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:46:29.470154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328d9e3-f1d8-47ae-b140-070526753f04 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:29.480677 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328d9e3-f1d8-47ae-b140-070526753f04 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:29.493018 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef00ab-8979-473a-90eb-cc1ca66f220b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:29.493568 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef00ab-8979-473a-90eb-cc1ca66f220b admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:29.493858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef00ab-8979-473a-90eb-cc1ca66f220b admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:29.494133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ef00ab-8979-473a-90eb-cc1ca66f220b admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:29.494805 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef00ab-8979-473a-90eb-cc1ca66f220b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:29.496867 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ef00ab-8979-473a-90eb-cc1ca66f220b admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:29.624622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1190/2399] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:46:29 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:46:29.628730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891e4b9-1e41-48c9-8c3d-d52b8f3511b3 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:29.638837 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d891e4b9-1e41-48c9-8c3d-d52b8f3511b3 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:29.696572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378071ee-d8a8-42f4-b8a8-42aae8168125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:29.697215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378071ee-d8a8-42f4-b8a8-42aae8168125 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:29.697534 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378071ee-d8a8-42f4-b8a8-42aae8168125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:29.697801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378071ee-d8a8-42f4-b8a8-42aae8168125 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:29.698466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378071ee-d8a8-42f4-b8a8-42aae8168125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:29.700455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378071ee-d8a8-42f4-b8a8-42aae8168125 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:29.759521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1210/2400] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:46:29 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:46:29.763491 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef00ab-8979-473a-90eb-cc1ca66f220b admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:29.774357 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ef00ab-8979-473a-90eb-cc1ca66f220b admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:29.779195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07647736-c81c-4117-a98e-87371bcf1c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:29.783340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07647736-c81c-4117-a98e-87371bcf1c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:29.783340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07647736-c81c-4117-a98e-87371bcf1c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:29.783340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07647736-c81c-4117-a98e-87371bcf1c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:29.783340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07647736-c81c-4117-a98e-87371bcf1c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:29.783684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07647736-c81c-4117-a98e-87371bcf1c5b admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:29.793434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1191/2401] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:46:29.797337 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378071ee-d8a8-42f4-b8a8-42aae8168125 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:29.808013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378071ee-d8a8-42f4-b8a8-42aae8168125 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:29.812710 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc2609-6e0e-416c-885e-48d15a730ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:29.816046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc2609-6e0e-416c-885e-48d15a730ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:29.816046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc2609-6e0e-416c-885e-48d15a730ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:29.816046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc2609-6e0e-416c-885e-48d15a730ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:29.816046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc2609-6e0e-416c-885e-48d15a730ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:29.816432 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc2609-6e0e-416c-885e-48d15a730ca6 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:29.827651 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1211/2402] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:46:29.831758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07647736-c81c-4117-a98e-87371bcf1c5b admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:29.841894 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07647736-c81c-4117-a98e-87371bcf1c5b admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:29.847296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade2055-6c58-4f4a-98f2-137f03ef0378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:29.847941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade2055-6c58-4f4a-98f2-137f03ef0378 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:29.848219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade2055-6c58-4f4a-98f2-137f03ef0378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:29.848490 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade2055-6c58-4f4a-98f2-137f03ef0378 admin admin] PATH_INFO: `/v3/projects/1c286736299f4d21bdbe8557491a7961/users/52d51a27875c45b0888e6b70a9f69e27/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:29.864212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade2055-6c58-4f4a-98f2-137f03ef0378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c286736299f4d21bdbe8557491a7961, user_id=52d51a27875c45b0888e6b70a9f69e27, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:29.866932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade2055-6c58-4f4a-98f2-137f03ef0378 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:29.937162 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1192/2403] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:46:29 2022] PUT /identity/v3/projects/1c286736299f4d21bdbe8557491a7961/users/52d51a27875c45b0888e6b70a9f69e27/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:46:29.947442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03009a09-8e6b-4b1c-b232-9b340a1b217a None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:29.947722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03009a09-8e6b-4b1c-b232-9b340a1b217a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:29.947990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03009a09-8e6b-4b1c-b232-9b340a1b217a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:29.976869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-03009a09-8e6b-4b1c-b232-9b340a1b217a None None] MFA Rules not processed for user `52d51a27875c45b0888e6b70a9f69e27`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:29.978113 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-03009a09-8e6b-4b1c-b232-9b340a1b217a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:30.040597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1212/2404] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:46:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:46:30.051992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade2055-6c58-4f4a-98f2-137f03ef0378 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:30.053672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7ade2055-6c58-4f4a-98f2-137f03ef0378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:30.076629 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:30.113688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade2055-6c58-4f4a-98f2-137f03ef0378 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:30.117331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3a1c4-29dd-4049-b3de-e3d560903a81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:30.121195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3a1c4-29dd-4049-b3de-e3d560903a81 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:30.121195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3a1c4-29dd-4049-b3de-e3d560903a81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:30.121195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3a1c4-29dd-4049-b3de-e3d560903a81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:30.127234 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3a1c4-29dd-4049-b3de-e3d560903a81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:30.128751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3a1c4-29dd-4049-b3de-e3d560903a81 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:30.138644 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1193/2405] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:46:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:46:30.352456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03009a09-8e6b-4b1c-b232-9b340a1b217a None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:30.356504 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-03009a09-8e6b-4b1c-b232-9b340a1b217a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:30.375397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:30.423034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03009a09-8e6b-4b1c-b232-9b340a1b217a None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:30.427434 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:30.428078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:30.428373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:30.428642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] PATH_INFO: `/v3/users/dc1aba6d01fe441aa740af05bc66074e` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:30.429778 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc1aba6d01fe441aa740af05bc66074e)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:30.431794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:30.591530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc1aba6d01fe441aa740af05bc66074e'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:30.592209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc1aba6d01fe441aa740af05bc66074e'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:30.690346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc1aba6d01fe441aa740af05bc66074e'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:30.690957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc1aba6d01fe441aa740af05bc66074e'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:30.698462 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc1aba6d01fe441aa740af05bc66074e'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:30.715481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1213/2406] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:46:30 2022] DELETE /identity/v3/users/dc1aba6d01fe441aa740af05bc66074e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:46:30.720087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580083f-c295-4f45-bac8-eee2e38acb41 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:30.720585 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580083f-c295-4f45-bac8-eee2e38acb41 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:30.720585 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580083f-c295-4f45-bac8-eee2e38acb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:30.749621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0580083f-c295-4f45-bac8-eee2e38acb41 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:30.751707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0580083f-c295-4f45-bac8-eee2e38acb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:30.828255 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1194/2407] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:46:30.838488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:30.840009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:30.908734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362006a-9f7f-4b54-9104-9ee85359c47e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:30.913390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30960b50-8ab9-45d0-9a37-9dc03cda189b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:30.914030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30960b50-8ab9-45d0-9a37-9dc03cda189b service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:30.914309 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30960b50-8ab9-45d0-9a37-9dc03cda189b service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:30.914583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30960b50-8ab9-45d0-9a37-9dc03cda189b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:30.924556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30960b50-8ab9-45d0-9a37-9dc03cda189b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:30.926038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30960b50-8ab9-45d0-9a37-9dc03cda189b service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:30.935535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1214/2408] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:46:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:46:31.467720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580083f-c295-4f45-bac8-eee2e38acb41 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:31.469247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0580083f-c295-4f45-bac8-eee2e38acb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:31.481922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580083f-c295-4f45-bac8-eee2e38acb41 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:31.486759 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3901fe-1429-4155-87ee-fc7b32693d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:31.487469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3901fe-1429-4155-87ee-fc7b32693d53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:31.487759 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3901fe-1429-4155-87ee-fc7b32693d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:31.488141 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3901fe-1429-4155-87ee-fc7b32693d53 admin admin] PATH_INFO: `/v3/projects/3bdb629ccf81494ebcc1192615e1187e` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:31.489249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3901fe-1429-4155-87ee-fc7b32693d53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bdb629ccf81494ebcc1192615e1187e)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:31.493970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3901fe-1429-4155-87ee-fc7b32693d53 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:31.621100 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6a3901fe-1429-4155-87ee-fc7b32693d53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bdb629ccf81494ebcc1192615e1187e', 'request_id': 'req-6a3901fe-1429-4155-87ee-fc7b32693d53'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:31.627173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6a3901fe-1429-4155-87ee-fc7b32693d53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bdb629ccf81494ebcc1192615e1187e', 'request_id': 'req-6a3901fe-1429-4155-87ee-fc7b32693d53'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:31.633340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6a3901fe-1429-4155-87ee-fc7b32693d53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bdb629ccf81494ebcc1192615e1187e', 'request_id': 'req-6a3901fe-1429-4155-87ee-fc7b32693d53'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:31.723294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6a3901fe-1429-4155-87ee-fc7b32693d53 admin admin] The token cache is being invalidate because project 3bdb629ccf81494ebcc1192615e1187e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:46:31.723952 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6a3901fe-1429-4155-87ee-fc7b32693d53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:31.724185 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-6a3901fe-1429-4155-87ee-fc7b32693d53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:31.725680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1195/2409] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:46:31 2022] DELETE /identity/v3/projects/3bdb629ccf81494ebcc1192615e1187e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:46:31.730727 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c10f5-3c93-41f6-8971-55229c001fca None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:31.740407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c10f5-3c93-41f6-8971-55229c001fca None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:31.740407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c10f5-3c93-41f6-8971-55229c001fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:31.750475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-d60c10f5-3c93-41f6-8971-55229c001fca None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:31.752390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d60c10f5-3c93-41f6-8971-55229c001fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:31.786023 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:31.810476 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1215/2410] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:46:31.818825 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4bea2-0e52-42e4-8ed1-87f02eb4bcbe None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:31.819106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4bea2-0e52-42e4-8ed1-87f02eb4bcbe None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:31.819370 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4bea2-0e52-42e4-8ed1-87f02eb4bcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:31.840882 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-bde4bea2-0e52-42e4-8ed1-87f02eb4bcbe None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:31.842129 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-bde4bea2-0e52-42e4-8ed1-87f02eb4bcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:31.851260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1196/2411] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:46:31.855749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c10f5-3c93-41f6-8971-55229c001fca None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:31.866257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c10f5-3c93-41f6-8971-55229c001fca None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:31.871183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb419bc-ca2b-4f5b-b5af-f8f4e5e5862b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:31.871801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb419bc-ca2b-4f5b-b5af-f8f4e5e5862b admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:31.872145 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb419bc-ca2b-4f5b-b5af-f8f4e5e5862b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:31.872490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb419bc-ca2b-4f5b-b5af-f8f4e5e5862b admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:31.873206 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb419bc-ca2b-4f5b-b5af-f8f4e5e5862b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:46:31.873575 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb419bc-ca2b-4f5b-b5af-f8f4e5e5862b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:31.875268 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb419bc-ca2b-4f5b-b5af-f8f4e5e5862b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:31.887784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1216/2412] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:46:31.890335 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26a882-2126-4d51-a231-3a3972410c8f None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:31.892798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26a882-2126-4d51-a231-3a3972410c8f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:31.892798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26a882-2126-4d51-a231-3a3972410c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:31.895965 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb419bc-ca2b-4f5b-b5af-f8f4e5e5862b admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:31.905911 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb419bc-ca2b-4f5b-b5af-f8f4e5e5862b admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:31.915536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0e26a882-2126-4d51-a231-3a3972410c8f None None] MFA Rules not processed for user `52d51a27875c45b0888e6b70a9f69e27`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:31.915728 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07783109-bd40-4e5d-8956-995ce2139851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:31.917185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07783109-bd40-4e5d-8956-995ce2139851 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:31.917310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0e26a882-2126-4d51-a231-3a3972410c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:31.917399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07783109-bd40-4e5d-8956-995ce2139851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:31.917399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07783109-bd40-4e5d-8956-995ce2139851 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:31.917952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07783109-bd40-4e5d-8956-995ce2139851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:31.919858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07783109-bd40-4e5d-8956-995ce2139851 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:31.951596 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:31.973378 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1197/2413] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:46:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:46:32.006198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1217/2414] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:46:31 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:46:32.010294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26a882-2126-4d51-a231-3a3972410c8f None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:32.021188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26a882-2126-4d51-a231-3a3972410c8f None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:32.087271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f15b9-4630-4b78-ba6b-9c68915c1bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:32.087889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f15b9-4630-4b78-ba6b-9c68915c1bad admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:32.088194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f15b9-4630-4b78-ba6b-9c68915c1bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:32.088465 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f15b9-4630-4b78-ba6b-9c68915c1bad admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:32.089104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f15b9-4630-4b78-ba6b-9c68915c1bad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:32.091077 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f15b9-4630-4b78-ba6b-9c68915c1bad admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:32.193515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1198/2415] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:46:32 2022] POST /identity/v3/users => generated 288 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:46:32.198070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07783109-bd40-4e5d-8956-995ce2139851 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:32.209341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07783109-bd40-4e5d-8956-995ce2139851 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:32.213988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c367e-3098-4d1f-8922-8b4b38cefc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:32.214588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c367e-3098-4d1f-8922-8b4b38cefc0d admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:32.214896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c367e-3098-4d1f-8922-8b4b38cefc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:32.215161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c367e-3098-4d1f-8922-8b4b38cefc0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:32.215800 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c367e-3098-4d1f-8922-8b4b38cefc0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:32.217388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c367e-3098-4d1f-8922-8b4b38cefc0d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:32.228589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1218/2416] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:46:32.232550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f15b9-4630-4b78-ba6b-9c68915c1bad admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:32.242837 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f15b9-4630-4b78-ba6b-9c68915c1bad admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:32.247681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b4edc-7ff8-4651-b9a2-0021cbdad7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:32.248347 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b4edc-7ff8-4651-b9a2-0021cbdad7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:32.248624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b4edc-7ff8-4651-b9a2-0021cbdad7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:32.248893 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b4edc-7ff8-4651-b9a2-0021cbdad7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:32.249530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b4edc-7ff8-4651-b9a2-0021cbdad7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:32.251441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b4edc-7ff8-4651-b9a2-0021cbdad7a4 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:32.261867 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1199/2417] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:46:32.266146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c367e-3098-4d1f-8922-8b4b38cefc0d admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:32.278248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c367e-3098-4d1f-8922-8b4b38cefc0d admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:32.283796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a19834-7a23-494c-9236-db955efd9e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:32.284446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a19834-7a23-494c-9236-db955efd9e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:32.284717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a19834-7a23-494c-9236-db955efd9e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:32.284985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a19834-7a23-494c-9236-db955efd9e83 admin admin] PATH_INFO: `/v3/projects/9eef7f31c18b49f49f245b33a2a2d639/users/d88beae72cb8430c9dbf881e4ae2aff6/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:32.300856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a19834-7a23-494c-9236-db955efd9e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eef7f31c18b49f49f245b33a2a2d639, user_id=d88beae72cb8430c9dbf881e4ae2aff6, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:32.303587 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a19834-7a23-494c-9236-db955efd9e83 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:32.373042 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1219/2418] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:46:32 2022] PUT /identity/v3/projects/9eef7f31c18b49f49f245b33a2a2d639/users/d88beae72cb8430c9dbf881e4ae2aff6/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:46:32.385209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af3275-4c2d-41fa-8611-1cc324c5fb3f None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:32.385430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af3275-4c2d-41fa-8611-1cc324c5fb3f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:32.385700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af3275-4c2d-41fa-8611-1cc324c5fb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:32.418720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-43af3275-4c2d-41fa-8611-1cc324c5fb3f None None] MFA Rules not processed for user `d88beae72cb8430c9dbf881e4ae2aff6`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:32.419926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-43af3275-4c2d-41fa-8611-1cc324c5fb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:32.488442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1200/2419] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:46:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:46:32.502093 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a19834-7a23-494c-9236-db955efd9e83 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:32.503965 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-06a19834-7a23-494c-9236-db955efd9e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:32.589646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a19834-7a23-494c-9236-db955efd9e83 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:32.593960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2638c7-28c1-47ed-b4c4-dd7802a2eba8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:32.594655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2638c7-28c1-47ed-b4c4-dd7802a2eba8 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:32.594935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2638c7-28c1-47ed-b4c4-dd7802a2eba8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:32.595207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2638c7-28c1-47ed-b4c4-dd7802a2eba8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:32.604800 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2638c7-28c1-47ed-b4c4-dd7802a2eba8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:32.606310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2638c7-28c1-47ed-b4c4-dd7802a2eba8 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:32.616857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1220/2420] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:46:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:46:36.676806 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af3275-4c2d-41fa-8611-1cc324c5fb3f None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:36.746489 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:36.758673 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af3275-4c2d-41fa-8611-1cc324c5fb3f None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:36.763770 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:36.764412 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:36.764680 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:36.764944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] PATH_INFO: `/v3/users/d88beae72cb8430c9dbf881e4ae2aff6` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:36.766099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d88beae72cb8430c9dbf881e4ae2aff6)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:36.768433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:37.023539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd88beae72cb8430c9dbf881e4ae2aff6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:37.026237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd88beae72cb8430c9dbf881e4ae2aff6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:37.141655 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd88beae72cb8430c9dbf881e4ae2aff6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:37.155933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd88beae72cb8430c9dbf881e4ae2aff6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:37.156957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd88beae72cb8430c9dbf881e4ae2aff6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:37.185978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1201/2421] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:46:36 2022] DELETE /identity/v3/users/d88beae72cb8430c9dbf881e4ae2aff6 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:46:37.189531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc37f1-c242-46cc-918d-7efb1b23714a None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:37.190029 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc37f1-c242-46cc-918d-7efb1b23714a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:37.190252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc37f1-c242-46cc-918d-7efb1b23714a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:37.215145 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-48fc37f1-c242-46cc-918d-7efb1b23714a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:37.216567 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-48fc37f1-c242-46cc-918d-7efb1b23714a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:37.295568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1221/2422] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:46:37.308563 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:37.310452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:37.416702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7925a8b1-2b51-4bbf-883e-d3181f642e02 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:37.420892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b3568-50b7-4d62-a0b6-8d974464a959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:37.421653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b3568-50b7-4d62-a0b6-8d974464a959 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:37.422074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b3568-50b7-4d62-a0b6-8d974464a959 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:37.422462 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b3568-50b7-4d62-a0b6-8d974464a959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:37.445765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b3568-50b7-4d62-a0b6-8d974464a959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:37.447431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b3568-50b7-4d62-a0b6-8d974464a959 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:37.458762 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1202/2423] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:46:38.066112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc37f1-c242-46cc-918d-7efb1b23714a None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:38.068401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-48fc37f1-c242-46cc-918d-7efb1b23714a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:38.082125 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc37f1-c242-46cc-918d-7efb1b23714a None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:38.091881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:38.092895 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:38.093237 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:38.093576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] PATH_INFO: `/v3/projects/9eef7f31c18b49f49f245b33a2a2d639` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:38.095116 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eef7f31c18b49f49f245b33a2a2d639)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:38.099878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:38.107785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:38.196444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eef7f31c18b49f49f245b33a2a2d639', 'request_id': 'req-e361d0d2-5fc8-4df7-b09d-256749307ce5'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:38.202541 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eef7f31c18b49f49f245b33a2a2d639', 'request_id': 'req-e361d0d2-5fc8-4df7-b09d-256749307ce5'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:38.221566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eef7f31c18b49f49f245b33a2a2d639', 'request_id': 'req-e361d0d2-5fc8-4df7-b09d-256749307ce5'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:38.348468 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] The token cache is being invalidate because project 9eef7f31c18b49f49f245b33a2a2d639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:46:38.349390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:38.350087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:46:38.351610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1222/2424] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:46:38 2022] DELETE /identity/v3/projects/9eef7f31c18b49f49f245b33a2a2d639 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:46:43.072290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3aba3a-36f1-4dba-9c13-838f1521dd9f None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:43.072811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3aba3a-36f1-4dba-9c13-838f1521dd9f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:43.072976 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3aba3a-36f1-4dba-9c13-838f1521dd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:43.099783 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-3c3aba3a-36f1-4dba-9c13-838f1521dd9f None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:43.101002 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3c3aba3a-36f1-4dba-9c13-838f1521dd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:43.158134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1203/2425] 10.222.0.49 () {58 vars in 967 bytes} [Fri Sep 16 05:46:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:46:43.181711 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:43.183121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:43.251085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361d0d2-5fc8-4df7-b09d-256749307ce5 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:43.256187 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8b770-4ab3-4660-8862-87709c74fba3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:43.256782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8b770-4ab3-4660-8862-87709c74fba3 service swift] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:43.257056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8b770-4ab3-4660-8862-87709c74fba3 service swift] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:43.257322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8b770-4ab3-4660-8862-87709c74fba3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:43.265322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8b770-4ab3-4660-8862-87709c74fba3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:43.266904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8b770-4ab3-4660-8862-87709c74fba3 service swift] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:43.275410 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1223/2426] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:46:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:46:47.939723 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed826236-5e56-4ba6-af88-5c7d4de615ec None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:47.940195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed826236-5e56-4ba6-af88-5c7d4de615ec None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:47.940291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed826236-5e56-4ba6-af88-5c7d4de615ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:47.959599 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-ed826236-5e56-4ba6-af88-5c7d4de615ec None None] MFA Rules not processed for user `2e593646e79c443490b22657996b9108`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:47.960803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-ed826236-5e56-4ba6-af88-5c7d4de615ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:47.973524 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:48.018305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1204/2427] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:46:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:46:53.272209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:58.159704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=16 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:46:58.214768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbfb90-b7d2-497f-b62b-05c2d3a8b7f8 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:58.215285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbfb90-b7d2-497f-b62b-05c2d3a8b7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:58.215285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbfb90-b7d2-497f-b62b-05c2d3a8b7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:58.235820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-8adbfb90-b7d2-497f-b62b-05c2d3a8b7f8 None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:58.237038 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8adbfb90-b7d2-497f-b62b-05c2d3a8b7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:58.244947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1224/2428] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:46:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:46:58.257257 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed826236-5e56-4ba6-af88-5c7d4de615ec None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:58.270201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed826236-5e56-4ba6-af88-5c7d4de615ec None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:58.275981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab5f58-3283-4eee-8346-c26277e347f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:58.276717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab5f58-3283-4eee-8346-c26277e347f1 service swift] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:58.279351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab5f58-3283-4eee-8346-c26277e347f1 service swift] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:58.279351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab5f58-3283-4eee-8346-c26277e347f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:58.286718 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab5f58-3283-4eee-8346-c26277e347f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:58.288281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab5f58-3283-4eee-8346-c26277e347f1 service swift] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:58.296767 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1205/2429] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:46:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 16 05:46:59.329606 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13078b-8837-44b1-b951-74b30d70cada None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:59.330129 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13078b-8837-44b1-b951-74b30d70cada None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:59.330492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13078b-8837-44b1-b951-74b30d70cada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:59.349464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-5e13078b-8837-44b1-b951-74b30d70cada None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:46:59.350690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5e13078b-8837-44b1-b951-74b30d70cada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:46:59.358030 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1225/2430] 10.222.0.49 () {58 vars in 967 bytes} [Fri Sep 16 05:46:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:46:59.371299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab5f58-3283-4eee-8346-c26277e347f1 service swift] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:46:59.381585 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab5f58-3283-4eee-8346-c26277e347f1 service swift] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:46:59.387209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d7cd8-0592-451b-8a6a-88976a467679 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:46:59.387798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d7cd8-0592-451b-8a6a-88976a467679 service swift] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:46:59.388072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d7cd8-0592-451b-8a6a-88976a467679 service swift] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:46:59.388339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994d7cd8-0592-451b-8a6a-88976a467679 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:46:59.396154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d7cd8-0592-451b-8a6a-88976a467679 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:46:59.397729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994d7cd8-0592-451b-8a6a-88976a467679 service swift] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:46:59.406351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1206/2431] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:46:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:47:00.004153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2e5c9-5254-409a-9fa3-35790bd1077e None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:00.004153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2e5c9-5254-409a-9fa3-35790bd1077e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:00.004153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2e5c9-5254-409a-9fa3-35790bd1077e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:00.020438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-25e2e5c9-5254-409a-9fa3-35790bd1077e None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:47:00.021555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-25e2e5c9-5254-409a-9fa3-35790bd1077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:00.029572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1226/2432] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:46:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:47:00.042412 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d7cd8-0592-451b-8a6a-88976a467679 service swift] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:47:00.052989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994d7cd8-0592-451b-8a6a-88976a467679 service swift] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:47:00.057984 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e723168-6a3d-4bc3-9669-76fd279f4943 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:47:00.058563 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e723168-6a3d-4bc3-9669-76fd279f4943 service swift] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:00.058833 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e723168-6a3d-4bc3-9669-76fd279f4943 service swift] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:00.059095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e723168-6a3d-4bc3-9669-76fd279f4943 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:00.067477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e723168-6a3d-4bc3-9669-76fd279f4943 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:47:00.069050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e723168-6a3d-4bc3-9669-76fd279f4943 service swift] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:47:00.077401 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1207/2433] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:47:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 16 05:47:00.630097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ab738-2acb-41d7-9f15-09735685a0e8 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:00.630588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ab738-2acb-41d7-9f15-09735685a0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:00.630694 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ab738-2acb-41d7-9f15-09735685a0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:00.649781 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-826ab738-2acb-41d7-9f15-09735685a0e8 None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:47:00.651708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-826ab738-2acb-41d7-9f15-09735685a0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:00.658917 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1227/2434] 10.222.0.49 () {58 vars in 967 bytes} [Fri Sep 16 05:47:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:47:00.671625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e723168-6a3d-4bc3-9669-76fd279f4943 service swift] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:47:00.686665 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e723168-6a3d-4bc3-9669-76fd279f4943 service swift] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:47:00.692526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa856089-b427-4508-aa4b-a3b813993a59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:47:00.693431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa856089-b427-4508-aa4b-a3b813993a59 service swift] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:00.693789 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa856089-b427-4508-aa4b-a3b813993a59 service swift] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:00.694095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa856089-b427-4508-aa4b-a3b813993a59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:00.702944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa856089-b427-4508-aa4b-a3b813993a59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:47:00.704514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa856089-b427-4508-aa4b-a3b813993a59 service swift] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:47:00.713133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1208/2435] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:47:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:47:03.378319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ab738-2acb-41d7-9f15-09735685a0e8 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:47:03.379822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-826ab738-2acb-41d7-9f15-09735685a0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:03.455581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ab738-2acb-41d7-9f15-09735685a0e8 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:47:03.460271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:47:03.460908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:03.461191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:03.461671 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] PATH_INFO: `/v3/users/52d51a27875c45b0888e6b70a9f69e27` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:03.463273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d51a27875c45b0888e6b70a9f69e27)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:47:03.465297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:47:03.666608 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d51a27875c45b0888e6b70a9f69e27'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:47:03.667243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d51a27875c45b0888e6b70a9f69e27'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:47:03.731588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d51a27875c45b0888e6b70a9f69e27'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:47:03.732154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d51a27875c45b0888e6b70a9f69e27'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:47:03.740229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d51a27875c45b0888e6b70a9f69e27'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:47:03.762689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1228/2436] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:47:03 2022] DELETE /identity/v3/users/52d51a27875c45b0888e6b70a9f69e27 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:47:03.767889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe88ad-ce73-461a-8cca-a02fb388b53a None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:03.768367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe88ad-ce73-461a-8cca-a02fb388b53a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:03.768488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe88ad-ce73-461a-8cca-a02fb388b53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:03.791115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f3fe88ad-ce73-461a-8cca-a02fb388b53a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:47:03.792504 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f3fe88ad-ce73-461a-8cca-a02fb388b53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:03.865798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1209/2437] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:47:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:47:03.880406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:47:03.882069 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:03.949954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91e210-d815-4d31-b112-e6521c9d11d2 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:47:03.953746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafcbfb2-cb5e-4994-95a2-9137fe2586a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:47:03.954390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcbfb2-cb5e-4994-95a2-9137fe2586a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:03.954668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcbfb2-cb5e-4994-95a2-9137fe2586a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:03.954937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcbfb2-cb5e-4994-95a2-9137fe2586a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:03.963128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:47:03.965618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafcbfb2-cb5e-4994-95a2-9137fe2586a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:47:03.967118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafcbfb2-cb5e-4994-95a2-9137fe2586a5 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:47:03.977383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1229/2438] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:47:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:47:04.236460 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe88ad-ce73-461a-8cca-a02fb388b53a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:47:04.236627 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f3fe88ad-ce73-461a-8cca-a02fb388b53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:04.257796 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe88ad-ce73-461a-8cca-a02fb388b53a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:47:04.274393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:47:04.274393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:04.274393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:04.274393 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] PATH_INFO: `/v3/projects/1c286736299f4d21bdbe8557491a7961` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:04.274722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c286736299f4d21bdbe8557491a7961)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:47:04.274722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:47:04.379801 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c286736299f4d21bdbe8557491a7961', 'request_id': 'req-84d1efcd-6160-4fb4-b158-11406d84721c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:47:04.387014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c286736299f4d21bdbe8557491a7961', 'request_id': 'req-84d1efcd-6160-4fb4-b158-11406d84721c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:47:04.393248 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c286736299f4d21bdbe8557491a7961', 'request_id': 'req-84d1efcd-6160-4fb4-b158-11406d84721c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:47:04.463837 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] The token cache is being invalidate because project 1c286736299f4d21bdbe8557491a7961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:47:04.464186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:47:04.464811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:47:04.466377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1210/2439] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:47:04 2022] DELETE /identity/v3/projects/1c286736299f4d21bdbe8557491a7961 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:47:11.493589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c29ef3-4385-4739-861f-bb00d00aca36 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:11.494040 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c29ef3-4385-4739-861f-bb00d00aca36 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:11.494198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c29ef3-4385-4739-861f-bb00d00aca36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:11.514148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-21c29ef3-4385-4739-861f-bb00d00aca36 None None] MFA Rules not processed for user `12e7e80742cc426da9984af75016a890`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:47:11.518139 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-21c29ef3-4385-4739-861f-bb00d00aca36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:11.569171 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1230/2440] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:47:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:47:14.459302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:47:21.562385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=27 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:47:35.190371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:47:35.192105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:35.264608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1efcd-6160-4fb4-b158-11406d84721c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:47:35.269299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb813120-d051-41fe-92ee-aa69c2469d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:47:35.270844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb813120-d051-41fe-92ee-aa69c2469d52 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:35.271149 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb813120-d051-41fe-92ee-aa69c2469d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:35.271498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb813120-d051-41fe-92ee-aa69c2469d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:35.274285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-cb813120-d051-41fe-92ee-aa69c2469d52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:35.355550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb813120-d051-41fe-92ee-aa69c2469d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:47:35.357580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb813120-d051-41fe-92ee-aa69c2469d52 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:47:35.369561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1211/2441] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:47:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:47:36.017722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f2e59-62ba-4856-8344-3f8d2a112724 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:36.018252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f2e59-62ba-4856-8344-3f8d2a112724 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:36.018357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f2e59-62ba-4856-8344-3f8d2a112724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:36.037311 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-3c2f2e59-62ba-4856-8344-3f8d2a112724 None None] MFA Rules not processed for user `c205ee9f85924da79b870648698caf64`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:47:36.043163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3c2f2e59-62ba-4856-8344-3f8d2a112724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:36.092797 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1231/2442] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:47:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:47:36.130835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb813120-d051-41fe-92ee-aa69c2469d52 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:47:36.132250 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-cb813120-d051-41fe-92ee-aa69c2469d52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:36.189434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb813120-d051-41fe-92ee-aa69c2469d52 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:47:36.193029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d0278-3274-45be-9cac-2eb0b37c209e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d96ec6235843c5b67d87a3d0ff0f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:47:36.193628 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d0278-3274-45be-9cac-2eb0b37c209e service glance] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:36.193941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d0278-3274-45be-9cac-2eb0b37c209e service glance] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:36.194210 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d0278-3274-45be-9cac-2eb0b37c209e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:36.195938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-626d0278-3274-45be-9cac-2eb0b37c209e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:36.209481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d0278-3274-45be-9cac-2eb0b37c209e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:47:36.210919 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d0278-3274-45be-9cac-2eb0b37c209e service glance] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:47:36.222512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1212/2443] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:47:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:47:46.086043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=25 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:47:46.217381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=80 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:47:52.144495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f2e59-62ba-4856-8344-3f8d2a112724 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:47:52.145861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3c2f2e59-62ba-4856-8344-3f8d2a112724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:52.162257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f2e59-62ba-4856-8344-3f8d2a112724 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:47:52.167701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c86417-3630-43fa-a279-df38dae41dd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:47:52.168342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c86417-3630-43fa-a279-df38dae41dd0 service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:52.168626 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c86417-3630-43fa-a279-df38dae41dd0 service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:52.168887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c86417-3630-43fa-a279-df38dae41dd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:52.170651 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e1c86417-3630-43fa-a279-df38dae41dd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:52.180818 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c86417-3630-43fa-a279-df38dae41dd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:47:52.182413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c86417-3630-43fa-a279-df38dae41dd0 service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:47:52.192391 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1232/2444] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:47:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:47:55.487135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d0278-3274-45be-9cac-2eb0b37c209e service glance] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:47:55.488529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-626d0278-3274-45be-9cac-2eb0b37c209e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:55.550321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d0278-3274-45be-9cac-2eb0b37c209e service glance] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:47:55.553958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77c4b3-d627-4d17-8d43-52485dbe5fbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:47:55.554554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77c4b3-d627-4d17-8d43-52485dbe5fbf service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:55.554823 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77c4b3-d627-4d17-8d43-52485dbe5fbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:55.555086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77c4b3-d627-4d17-8d43-52485dbe5fbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:55.556775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-eb77c4b3-d627-4d17-8d43-52485dbe5fbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:55.616083 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77c4b3-d627-4d17-8d43-52485dbe5fbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:47:55.617548 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77c4b3-d627-4d17-8d43-52485dbe5fbf service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:47:55.628108 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1213/2445] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:47:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:47:58.020346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c86417-3630-43fa-a279-df38dae41dd0 service nova] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:47:58.021743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e1c86417-3630-43fa-a279-df38dae41dd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:58.034313 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c86417-3630-43fa-a279-df38dae41dd0 service nova] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:47:58.037867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947874e9-bfe6-49e3-8aff-b1422a02f58e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:47:58.038436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947874e9-bfe6-49e3-8aff-b1422a02f58e service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:47:58.038714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947874e9-bfe6-49e3-8aff-b1422a02f58e service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:47:58.038981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947874e9-bfe6-49e3-8aff-b1422a02f58e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:47:58.040695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-947874e9-bfe6-49e3-8aff-b1422a02f58e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:47:58.052530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947874e9-bfe6-49e3-8aff-b1422a02f58e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:47:58.054065 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947874e9-bfe6-49e3-8aff-b1422a02f58e service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:47:58.063452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1233/2446] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:47:58 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:48:04.239501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316fa72-1a03-44b0-b433-9e9e7ce1acea None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:04.240683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316fa72-1a03-44b0-b433-9e9e7ce1acea None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:04.240683 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316fa72-1a03-44b0-b433-9e9e7ce1acea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:04.259792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-8316fa72-1a03-44b0-b433-9e9e7ce1acea None None] MFA Rules not processed for user `edd619bf865645fab0460d3aa847a734`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:48:04.263325 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8316fa72-1a03-44b0-b433-9e9e7ce1acea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:04.268817 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1214/2447] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:48:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:48:04.475935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947874e9-bfe6-49e3-8aff-b1422a02f58e service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:04.478747 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-947874e9-bfe6-49e3-8aff-b1422a02f58e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:04.558575 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947874e9-bfe6-49e3-8aff-b1422a02f58e service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:04.563575 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:04.564531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:04.564986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:04.565260 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] PATH_INFO: `/v3/users/2e593646e79c443490b22657996b9108` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:04.566436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e593646e79c443490b22657996b9108)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:04.568350 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:04.614392 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316fa72-1a03-44b0-b433-9e9e7ce1acea None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:04.616435 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8316fa72-1a03-44b0-b433-9e9e7ce1acea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:04.629153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316fa72-1a03-44b0-b433-9e9e7ce1acea None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:04.633117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ee4b2-7cea-4e4c-a02f-d1cfe0c04822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:04.633851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ee4b2-7cea-4e4c-a02f-d1cfe0c04822 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:04.634124 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ee4b2-7cea-4e4c-a02f-d1cfe0c04822 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:04.634392 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ee4b2-7cea-4e4c-a02f-d1cfe0c04822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:04.636107 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-960ee4b2-7cea-4e4c-a02f-d1cfe0c04822 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:04.646516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ee4b2-7cea-4e4c-a02f-d1cfe0c04822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:04.649036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ee4b2-7cea-4e4c-a02f-d1cfe0c04822 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:04.656381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1215/2448] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:48:04.768528 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e593646e79c443490b22657996b9108'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:04.769486 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e593646e79c443490b22657996b9108'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:04.841185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e593646e79c443490b22657996b9108'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:04.841920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e593646e79c443490b22657996b9108'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:04.851501 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e593646e79c443490b22657996b9108'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:04.881322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1234/2449] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:48:04 2022] DELETE /identity/v3/users/2e593646e79c443490b22657996b9108 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:48:05.342032 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ee4b2-7cea-4e4c-a02f-d1cfe0c04822 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:05.344951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:05.347519 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:05.364597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ee4b2-7cea-4e4c-a02f-d1cfe0c04822 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:05.442211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:48:05.452277 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cacb7df-fe15-4da9-935a-7f3a977e0008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:05.453843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacb7df-fe15-4da9-935a-7f3a977e0008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:05.454579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacb7df-fe15-4da9-935a-7f3a977e0008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:05.455172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cacb7df-fe15-4da9-935a-7f3a977e0008 admin admin] PATH_INFO: `/v3/projects/04dc3f5e68e046fb9bf360d4f01a50fd` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:05.456478 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cacb7df-fe15-4da9-935a-7f3a977e0008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04dc3f5e68e046fb9bf360d4f01a50fd)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:05.457452 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07860921-3a3b-4875-a4e6-26886718bdfe admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:05.458813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cacb7df-fe15-4da9-935a-7f3a977e0008 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:05.463490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d51c7b-f084-4e72-9253-1c919edd759f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9975c0dab76f4ccbb4d2107e8e780ae5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:05.464440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d51c7b-f084-4e72-9253-1c919edd759f service placement] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:05.464905 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d51c7b-f084-4e72-9253-1c919edd759f service placement] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:05.469690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d51c7b-f084-4e72-9253-1c919edd759f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:05.486863 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a9d51c7b-f084-4e72-9253-1c919edd759f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:05.503647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d51c7b-f084-4e72-9253-1c919edd759f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:05.505385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d51c7b-f084-4e72-9253-1c919edd759f service placement] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:05.520583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1235/2450] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:48:05.623040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:48:05.632946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8cacb7df-fe15-4da9-935a-7f3a977e0008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04dc3f5e68e046fb9bf360d4f01a50fd', 'request_id': 'req-8cacb7df-fe15-4da9-935a-7f3a977e0008'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:05.640780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8cacb7df-fe15-4da9-935a-7f3a977e0008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04dc3f5e68e046fb9bf360d4f01a50fd', 'request_id': 'req-8cacb7df-fe15-4da9-935a-7f3a977e0008'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:05.649614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8cacb7df-fe15-4da9-935a-7f3a977e0008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04dc3f5e68e046fb9bf360d4f01a50fd', 'request_id': 'req-8cacb7df-fe15-4da9-935a-7f3a977e0008'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:05.722330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8cacb7df-fe15-4da9-935a-7f3a977e0008 admin admin] The token cache is being invalidate because project 04dc3f5e68e046fb9bf360d4f01a50fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:48:05.722707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8cacb7df-fe15-4da9-935a-7f3a977e0008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:05.723358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8cacb7df-fe15-4da9-935a-7f3a977e0008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:05.728374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1216/2451] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:48:05 2022] DELETE /identity/v3/projects/04dc3f5e68e046fb9bf360d4f01a50fd => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:48:05.730171 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247f192-5254-46a0-95c8-7cfc3960e6c9 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:05.730560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247f192-5254-46a0-95c8-7cfc3960e6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:05.730966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247f192-5254-46a0-95c8-7cfc3960e6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:05.758443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-9247f192-5254-46a0-95c8-7cfc3960e6c9 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:48:05.759744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-9247f192-5254-46a0-95c8-7cfc3960e6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:05.844550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1236/2452] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:48:05.847235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784ba7a-6f43-4214-b5d9-62f064a45cb9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:05.847235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784ba7a-6f43-4214-b5d9-62f064a45cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:05.847235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784ba7a-6f43-4214-b5d9-62f064a45cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:05.877315 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-e784ba7a-6f43-4214-b5d9-62f064a45cb9 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:48:05.878672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e784ba7a-6f43-4214-b5d9-62f064a45cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:05.897238 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1217/2453] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:48:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:48:05.898531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247f192-5254-46a0-95c8-7cfc3960e6c9 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:05.916864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247f192-5254-46a0-95c8-7cfc3960e6c9 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:05.926019 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3336821-b469-4140-a548-0eeaeefad753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:05.926185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3336821-b469-4140-a548-0eeaeefad753 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:05.926443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3336821-b469-4140-a548-0eeaeefad753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:05.926780 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3336821-b469-4140-a548-0eeaeefad753 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:05.927303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3336821-b469-4140-a548-0eeaeefad753 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:48:05.927597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3336821-b469-4140-a548-0eeaeefad753 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:05.929746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3336821-b469-4140-a548-0eeaeefad753 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:05.951203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1237/2454] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:48:05.952715 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784ba7a-6f43-4214-b5d9-62f064a45cb9 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:05.962514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784ba7a-6f43-4214-b5d9-62f064a45cb9 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:05.968554 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78c01b-db94-42ea-b7b5-4ad4752d1686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:05.969227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78c01b-db94-42ea-b7b5-4ad4752d1686 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:05.969511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78c01b-db94-42ea-b7b5-4ad4752d1686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:05.969857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78c01b-db94-42ea-b7b5-4ad4752d1686 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:05.970853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78c01b-db94-42ea-b7b5-4ad4752d1686 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:05.972943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea78c01b-db94-42ea-b7b5-4ad4752d1686 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:06.060619 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1218/2455] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:48:05 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:48:06.066096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3336821-b469-4140-a548-0eeaeefad753 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:06.077599 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3336821-b469-4140-a548-0eeaeefad753 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:06.168286 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dcc10e-49cc-4007-b498-4e3f0f34e261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:06.168899 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcc10e-49cc-4007-b498-4e3f0f34e261 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:06.169174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcc10e-49cc-4007-b498-4e3f0f34e261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:06.169442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcc10e-49cc-4007-b498-4e3f0f34e261 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:06.170115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dcc10e-49cc-4007-b498-4e3f0f34e261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:06.172044 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dcc10e-49cc-4007-b498-4e3f0f34e261 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:06.264852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1238/2456] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:48:06 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:48:06.270089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78c01b-db94-42ea-b7b5-4ad4752d1686 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:06.282103 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78c01b-db94-42ea-b7b5-4ad4752d1686 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:06.291005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762a2c1-5668-48b9-810e-d7a3da8a6b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:06.291656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762a2c1-5668-48b9-810e-d7a3da8a6b53 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:06.297579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762a2c1-5668-48b9-810e-d7a3da8a6b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:06.297579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762a2c1-5668-48b9-810e-d7a3da8a6b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:06.297579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762a2c1-5668-48b9-810e-d7a3da8a6b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:06.297579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762a2c1-5668-48b9-810e-d7a3da8a6b53 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:06.318934 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1219/2457] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:48:06.323141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dcc10e-49cc-4007-b498-4e3f0f34e261 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:06.335847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dcc10e-49cc-4007-b498-4e3f0f34e261 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:06.340577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f911fcc-7f20-46b6-96ca-9d2e4a214268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:06.341208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f911fcc-7f20-46b6-96ca-9d2e4a214268 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:06.341572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f911fcc-7f20-46b6-96ca-9d2e4a214268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:06.341918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f911fcc-7f20-46b6-96ca-9d2e4a214268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:06.342572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f911fcc-7f20-46b6-96ca-9d2e4a214268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:06.344164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f911fcc-7f20-46b6-96ca-9d2e4a214268 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:06.359140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1239/2458] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:48:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:48:06.363664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762a2c1-5668-48b9-810e-d7a3da8a6b53 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:06.373999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762a2c1-5668-48b9-810e-d7a3da8a6b53 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:06.379154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ac308-5c82-4387-ae0e-fdb9d8bce862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:06.379961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac308-5c82-4387-ae0e-fdb9d8bce862 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:06.380236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac308-5c82-4387-ae0e-fdb9d8bce862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:06.380504 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac308-5c82-4387-ae0e-fdb9d8bce862 admin admin] PATH_INFO: `/v3/projects/65eddf04dea24b1ba57c733b822888a8/users/1f15b226ef0e45229d55f32b38c0386f/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:06.396457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ac308-5c82-4387-ae0e-fdb9d8bce862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65eddf04dea24b1ba57c733b822888a8, user_id=1f15b226ef0e45229d55f32b38c0386f, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:06.399172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ac308-5c82-4387-ae0e-fdb9d8bce862 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:06.500035 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1220/2459] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:48:06 2022] PUT /identity/v3/projects/65eddf04dea24b1ba57c733b822888a8/users/1f15b226ef0e45229d55f32b38c0386f/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:48:06.513853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001eec-4a63-4b6e-af4e-09d20b3502ea None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:06.513853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001eec-4a63-4b6e-af4e-09d20b3502ea None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:06.514439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001eec-4a63-4b6e-af4e-09d20b3502ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:06.559563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-8b001eec-4a63-4b6e-af4e-09d20b3502ea None None] MFA Rules not processed for user `1f15b226ef0e45229d55f32b38c0386f`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:48:06.560885 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8b001eec-4a63-4b6e-af4e-09d20b3502ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:06.605075 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:48:06.635148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1240/2460] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:48:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:48:06.647528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ac308-5c82-4387-ae0e-fdb9d8bce862 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:06.649216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6b1ac308-5c82-4387-ae0e-fdb9d8bce862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:06.710317 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ac308-5c82-4387-ae0e-fdb9d8bce862 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:06.714104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc84b7b-a95a-42cf-9802-83ed1d742067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:06.714721 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc84b7b-a95a-42cf-9802-83ed1d742067 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:06.715005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc84b7b-a95a-42cf-9802-83ed1d742067 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:06.715272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc84b7b-a95a-42cf-9802-83ed1d742067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:06.726777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc84b7b-a95a-42cf-9802-83ed1d742067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:06.728226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc84b7b-a95a-42cf-9802-83ed1d742067 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:06.738919 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1221/2461] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:48:09.335986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b001eec-4a63-4b6e-af4e-09d20b3502ea None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:09.335986 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8b001eec-4a63-4b6e-af4e-09d20b3502ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:09.391261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b001eec-4a63-4b6e-af4e-09d20b3502ea None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:09.395267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e46a4-42ce-4615-bcd4-521e2bbd7420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:09.395877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e46a4-42ce-4615-bcd4-521e2bbd7420 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:09.396155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e46a4-42ce-4615-bcd4-521e2bbd7420 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:09.396424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e46a4-42ce-4615-bcd4-521e2bbd7420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:09.398192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-783e46a4-42ce-4615-bcd4-521e2bbd7420 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:09.408725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e46a4-42ce-4615-bcd4-521e2bbd7420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:09.410492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e46a4-42ce-4615-bcd4-521e2bbd7420 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:09.420679 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1241/2462] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:48:10.291244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc84b7b-a95a-42cf-9802-83ed1d742067 service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:10.292669 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2fc84b7b-a95a-42cf-9802-83ed1d742067 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:10.363764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:48:10.369780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc84b7b-a95a-42cf-9802-83ed1d742067 service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:10.375258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d48c72-ac88-479e-a728-74bf2adebf0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:10.375849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d48c72-ac88-479e-a728-74bf2adebf0f service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:10.376125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d48c72-ac88-479e-a728-74bf2adebf0f service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:10.376384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d48c72-ac88-479e-a728-74bf2adebf0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:10.378785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-59d48c72-ac88-479e-a728-74bf2adebf0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:10.390658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d48c72-ac88-479e-a728-74bf2adebf0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:10.392257 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d48c72-ac88-479e-a728-74bf2adebf0f service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:10.402981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1222/2463] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:48:19.416543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=36 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:48:20.451788 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=5 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:48:25.433655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e46a4-42ce-4615-bcd4-521e2bbd7420 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:25.451015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e46a4-42ce-4615-bcd4-521e2bbd7420 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:25.457000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dcc70-7753-48b7-ba88-b398529a30fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:25.457609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dcc70-7753-48b7-ba88-b398529a30fd service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:25.458617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dcc70-7753-48b7-ba88-b398529a30fd service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:25.459002 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dcc70-7753-48b7-ba88-b398529a30fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:25.460809 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d65dcc70-7753-48b7-ba88-b398529a30fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:25.476208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dcc70-7753-48b7-ba88-b398529a30fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:25.477700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dcc70-7753-48b7-ba88-b398529a30fd service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:25.489854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1242/2464] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:48:27.696517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d48c72-ac88-479e-a728-74bf2adebf0f service nova] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:27.763949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d48c72-ac88-479e-a728-74bf2adebf0f service nova] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:27.764829 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:27.765456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:27.765758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:27.766168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] PATH_INFO: `/v3/users/1f15b226ef0e45229d55f32b38c0386f` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:27.767160 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f15b226ef0e45229d55f32b38c0386f)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:27.769093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:28.000787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f15b226ef0e45229d55f32b38c0386f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:28.001439 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f15b226ef0e45229d55f32b38c0386f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:28.066270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f15b226ef0e45229d55f32b38c0386f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:28.074158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f15b226ef0e45229d55f32b38c0386f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:28.077021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f15b226ef0e45229d55f32b38c0386f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:28.093084 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1223/2465] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:48:27 2022] DELETE /identity/v3/users/1f15b226ef0e45229d55f32b38c0386f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:48:28.097613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eee0b-d8c1-47b1-9c77-02cd8774c6e1 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:28.098146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eee0b-d8c1-47b1-9c77-02cd8774c6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:28.098253 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eee0b-d8c1-47b1-9c77-02cd8774c6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:28.118815 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-af5eee0b-d8c1-47b1-9c77-02cd8774c6e1 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:48:28.120003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-af5eee0b-d8c1-47b1-9c77-02cd8774c6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:28.178830 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1243/2466] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:48:28.188723 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:28.190139 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:28.253236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef167f-862d-44a0-be1d-06ae4daea4a2 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:28.257216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dc565-3e44-41bd-bfc0-a4ddb3875c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:28.258014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dc565-3e44-41bd-bfc0-a4ddb3875c55 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:28.258301 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dc565-3e44-41bd-bfc0-a4ddb3875c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:28.258651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dc565-3e44-41bd-bfc0-a4ddb3875c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:28.268606 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45dc565-3e44-41bd-bfc0-a4ddb3875c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:28.270970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45dc565-3e44-41bd-bfc0-a4ddb3875c55 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:28.281466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1224/2467] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:48:28.870776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eee0b-d8c1-47b1-9c77-02cd8774c6e1 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:28.872601 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-af5eee0b-d8c1-47b1-9c77-02cd8774c6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:28.885602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eee0b-d8c1-47b1-9c77-02cd8774c6e1 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:28.890475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:28.891285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:28.891731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:28.892201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] PATH_INFO: `/v3/projects/65eddf04dea24b1ba57c733b822888a8` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:28.893523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65eddf04dea24b1ba57c733b822888a8)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:28.895518 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:29.016830 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65eddf04dea24b1ba57c733b822888a8', 'request_id': 'req-aaff3e77-65bb-40dd-9de9-6a931fe42df9'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:29.026065 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65eddf04dea24b1ba57c733b822888a8', 'request_id': 'req-aaff3e77-65bb-40dd-9de9-6a931fe42df9'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:29.033917 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65eddf04dea24b1ba57c733b822888a8', 'request_id': 'req-aaff3e77-65bb-40dd-9de9-6a931fe42df9'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:29.135158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] The token cache is being invalidate because project 65eddf04dea24b1ba57c733b822888a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:48:29.135844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:29.136131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:29.137838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1244/2468] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:48:28 2022] DELETE /identity/v3/projects/65eddf04dea24b1ba57c733b822888a8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:48:32.858188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dc565-3e44-41bd-bfc0-a4ddb3875c55 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:32.866776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dc565-3e44-41bd-bfc0-a4ddb3875c55 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:32.886047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:48:32.920448 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc1664-7eed-41f3-9aca-63dfd08b667a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:32.921198 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc1664-7eed-41f3-9aca-63dfd08b667a service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:32.921592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc1664-7eed-41f3-9aca-63dfd08b667a service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:32.921926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc1664-7eed-41f3-9aca-63dfd08b667a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:32.924006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a1bc1664-7eed-41f3-9aca-63dfd08b667a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:32.962511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:32.964047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:32.977093 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff3e77-65bb-40dd-9de9-6a931fe42df9 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:32.980646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcf1e1-2b10-49e5-bb1c-ed758d39812f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:32.981367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcf1e1-2b10-49e5-bb1c-ed758d39812f service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:32.981640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcf1e1-2b10-49e5-bb1c-ed758d39812f service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:32.981938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcf1e1-2b10-49e5-bb1c-ed758d39812f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:32.984192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b4fcf1e1-2b10-49e5-bb1c-ed758d39812f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:32.991832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc1664-7eed-41f3-9aca-63dfd08b667a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:32.993273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc1664-7eed-41f3-9aca-63dfd08b667a service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:32.994520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcf1e1-2b10-49e5-bb1c-ed758d39812f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:32.995950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcf1e1-2b10-49e5-bb1c-ed758d39812f service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:33.003679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1225/2469] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:48:33.004547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1245/2470] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:48:33.405236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44bc29-43d8-4043-ba4e-0d8a03d26a42 None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:33.405613 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44bc29-43d8-4043-ba4e-0d8a03d26a42 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:33.406172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44bc29-43d8-4043-ba4e-0d8a03d26a42 None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:33.407576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1226/2471] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:48:33.414174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c9bae-9b3d-4b82-9f4f-28ffaebba896 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:33.414459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c9bae-9b3d-4b82-9f4f-28ffaebba896 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:33.414731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c9bae-9b3d-4b82-9f4f-28ffaebba896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:33.436964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-216c9bae-9b3d-4b82-9f4f-28ffaebba896 None None] MFA Rules not processed for user `3e923be9aeed43728690d24d85dbe062`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:48:33.438128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-216c9bae-9b3d-4b82-9f4f-28ffaebba896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:33.445437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1246/2472] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:48:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:48:33.458365 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44bc29-43d8-4043-ba4e-0d8a03d26a42 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:33.469872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44bc29-43d8-4043-ba4e-0d8a03d26a42 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:33.473510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149791d8-4ab0-4ef0-aefd-dd1344ed88fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:33.474226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149791d8-4ab0-4ef0-aefd-dd1344ed88fc service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:33.474565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149791d8-4ab0-4ef0-aefd-dd1344ed88fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:33.474903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149791d8-4ab0-4ef0-aefd-dd1344ed88fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:33.482854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149791d8-4ab0-4ef0-aefd-dd1344ed88fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:33.484382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149791d8-4ab0-4ef0-aefd-dd1344ed88fc service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:33.493171 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1227/2473] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:48:33.581176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34911c73-c132-4510-afed-ea42aa5bc4dc None None] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:33.581455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34911c73-c132-4510-afed-ea42aa5bc4dc None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:33.581725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34911c73-c132-4510-afed-ea42aa5bc4dc None None] PATH_INFO: `/` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:33.583646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1247/2474] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:48:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:48:33.588883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637f719-516f-4115-80bb-0f300145f9e4 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:33.589166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637f719-516f-4115-80bb-0f300145f9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:33.589438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637f719-516f-4115-80bb-0f300145f9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:33.610872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-4637f719-516f-4115-80bb-0f300145f9e4 None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:48:33.612176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4637f719-516f-4115-80bb-0f300145f9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:33.683723 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1228/2475] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:48:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:48:33.695164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34911c73-c132-4510-afed-ea42aa5bc4dc None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:33.706591 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34911c73-c132-4510-afed-ea42aa5bc4dc None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:33.711186 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa111126-3810-40c3-a66e-5a39d05c14c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:33.711787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa111126-3810-40c3-a66e-5a39d05c14c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:33.712058 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa111126-3810-40c3-a66e-5a39d05c14c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:33.712325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa111126-3810-40c3-a66e-5a39d05c14c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:33.725391 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa111126-3810-40c3-a66e-5a39d05c14c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:33.726851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa111126-3810-40c3-a66e-5a39d05c14c6 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:33.737951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1248/2476] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:48:34.393774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637f719-516f-4115-80bb-0f300145f9e4 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:34.395399 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4637f719-516f-4115-80bb-0f300145f9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:34.413171 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637f719-516f-4115-80bb-0f300145f9e4 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:34.420313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352d627-db35-41c3-bc7d-443a1e54677c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:34.421126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352d627-db35-41c3-bc7d-443a1e54677c service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:34.421494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352d627-db35-41c3-bc7d-443a1e54677c service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:34.421889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352d627-db35-41c3-bc7d-443a1e54677c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:34.425581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a352d627-db35-41c3-bc7d-443a1e54677c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:34.440930 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352d627-db35-41c3-bc7d-443a1e54677c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:34.442770 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352d627-db35-41c3-bc7d-443a1e54677c service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:34.455425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1229/2477] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:48:37.060552 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa111126-3810-40c3-a66e-5a39d05c14c6 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:37.062076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-aa111126-3810-40c3-a66e-5a39d05c14c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:37.077492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa111126-3810-40c3-a66e-5a39d05c14c6 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:37.082376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877680d3-f6fb-4fff-a163-a4d8451fa5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:37.083665 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877680d3-f6fb-4fff-a163-a4d8451fa5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:37.083975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877680d3-f6fb-4fff-a163-a4d8451fa5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:37.084248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877680d3-f6fb-4fff-a163-a4d8451fa5f9 admin admin] PATH_INFO: `/v3/users/12e7e80742cc426da9984af75016a890` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:37.085403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877680d3-f6fb-4fff-a163-a4d8451fa5f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e7e80742cc426da9984af75016a890)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:37.087460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877680d3-f6fb-4fff-a163-a4d8451fa5f9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:37.228372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:48:37.240578 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-877680d3-f6fb-4fff-a163-a4d8451fa5f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e7e80742cc426da9984af75016a890'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:37.241229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-877680d3-f6fb-4fff-a163-a4d8451fa5f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e7e80742cc426da9984af75016a890'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:37.326713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-877680d3-f6fb-4fff-a163-a4d8451fa5f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e7e80742cc426da9984af75016a890'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:37.327310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-877680d3-f6fb-4fff-a163-a4d8451fa5f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e7e80742cc426da9984af75016a890'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:37.336666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-877680d3-f6fb-4fff-a163-a4d8451fa5f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e7e80742cc426da9984af75016a890'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:37.354768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1249/2478] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:48:37 2022] DELETE /identity/v3/users/12e7e80742cc426da9984af75016a890 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:48:37.670662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352d627-db35-41c3-bc7d-443a1e54677c service nova] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:37.682146 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352d627-db35-41c3-bc7d-443a1e54677c service nova] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:37.720322 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:48:37.743308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8866a-1b3a-4145-8259-85b78bad04b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:37.743941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8866a-1b3a-4145-8259-85b78bad04b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:37.744235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8866a-1b3a-4145-8259-85b78bad04b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:37.744486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8866a-1b3a-4145-8259-85b78bad04b6 admin admin] PATH_INFO: `/v3/projects/d6a0481b87f34225a6a44f702ba75f68` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:37.745620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8866a-1b3a-4145-8259-85b78bad04b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6a0481b87f34225a6a44f702ba75f68)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:37.747594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8866a-1b3a-4145-8259-85b78bad04b6 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:37.860283 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11f8866a-1b3a-4145-8259-85b78bad04b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6a0481b87f34225a6a44f702ba75f68', 'request_id': 'req-11f8866a-1b3a-4145-8259-85b78bad04b6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:37.865863 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11f8866a-1b3a-4145-8259-85b78bad04b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6a0481b87f34225a6a44f702ba75f68', 'request_id': 'req-11f8866a-1b3a-4145-8259-85b78bad04b6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:37.871944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11f8866a-1b3a-4145-8259-85b78bad04b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6a0481b87f34225a6a44f702ba75f68', 'request_id': 'req-11f8866a-1b3a-4145-8259-85b78bad04b6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:37.944477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11f8866a-1b3a-4145-8259-85b78bad04b6 admin admin] The token cache is being invalidate because project d6a0481b87f34225a6a44f702ba75f68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:48:37.944763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11f8866a-1b3a-4145-8259-85b78bad04b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:37.945440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-11f8866a-1b3a-4145-8259-85b78bad04b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:48:37.946938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1230/2479] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:48:37 2022] DELETE /identity/v3/projects/d6a0481b87f34225a6a44f702ba75f68 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:48:37.951994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb9e7b-b986-43dd-a0f9-1e292107b64b None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:37.952277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb9e7b-b986-43dd-a0f9-1e292107b64b None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:37.952545 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb9e7b-b986-43dd-a0f9-1e292107b64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:37.972758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-80fb9e7b-b986-43dd-a0f9-1e292107b64b None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:48:37.973933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-80fb9e7b-b986-43dd-a0f9-1e292107b64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:38.031396 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1250/2480] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:48:38.039809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e2128-1494-4121-8134-031174b074d9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:38.040120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e2128-1494-4121-8134-031174b074d9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:38.040388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e2128-1494-4121-8134-031174b074d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:38.061034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-b35e2128-1494-4121-8134-031174b074d9 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:48:38.062275 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b35e2128-1494-4121-8134-031174b074d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:38.071075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1231/2481] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:48:38.075319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb9e7b-b986-43dd-a0f9-1e292107b64b None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:38.084940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb9e7b-b986-43dd-a0f9-1e292107b64b None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:38.089492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda61ad-ba3f-4ad2-8860-c9d89817871e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:38.090308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda61ad-ba3f-4ad2-8860-c9d89817871e admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:38.090591 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda61ad-ba3f-4ad2-8860-c9d89817871e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:38.090861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda61ad-ba3f-4ad2-8860-c9d89817871e admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:38.091530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda61ad-ba3f-4ad2-8860-c9d89817871e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:48:38.091851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda61ad-ba3f-4ad2-8860-c9d89817871e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:38.093435 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda61ad-ba3f-4ad2-8860-c9d89817871e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:38.105087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1251/2482] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:48:38.109269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e2128-1494-4121-8134-031174b074d9 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:38.119040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e2128-1494-4121-8134-031174b074d9 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:38.123643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4028212-f614-431a-a6c5-50c971c52c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:38.124235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4028212-f614-431a-a6c5-50c971c52c09 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:38.124506 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4028212-f614-431a-a6c5-50c971c52c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:38.124769 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4028212-f614-431a-a6c5-50c971c52c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:38.125400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4028212-f614-431a-a6c5-50c971c52c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:38.127282 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4028212-f614-431a-a6c5-50c971c52c09 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:38.228404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1232/2483] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:48:38 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:48:38.229529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda61ad-ba3f-4ad2-8860-c9d89817871e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:38.240573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda61ad-ba3f-4ad2-8860-c9d89817871e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:38.296659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f148c-02ac-4c6c-a151-3ea0ca540592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:38.297264 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f148c-02ac-4c6c-a151-3ea0ca540592 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:38.297541 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f148c-02ac-4c6c-a151-3ea0ca540592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:38.297809 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f148c-02ac-4c6c-a151-3ea0ca540592 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:38.298487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f148c-02ac-4c6c-a151-3ea0ca540592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:38.300388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f148c-02ac-4c6c-a151-3ea0ca540592 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:38.389658 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1252/2484] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:48:38 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:48:38.393922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4028212-f614-431a-a6c5-50c971c52c09 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:38.404327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4028212-f614-431a-a6c5-50c971c52c09 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:38.409275 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57159f-3746-43a3-9d3b-91018df3bfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:38.409891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57159f-3746-43a3-9d3b-91018df3bfde admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:38.410168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57159f-3746-43a3-9d3b-91018df3bfde admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:38.410432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57159f-3746-43a3-9d3b-91018df3bfde admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:38.411617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57159f-3746-43a3-9d3b-91018df3bfde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:38.413371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57159f-3746-43a3-9d3b-91018df3bfde admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:38.423354 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1233/2485] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:48:38.427850 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f148c-02ac-4c6c-a151-3ea0ca540592 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:38.437625 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f148c-02ac-4c6c-a151-3ea0ca540592 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:38.442243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec3f65-659d-4c7c-82b8-7083ff21dbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:38.442838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3f65-659d-4c7c-82b8-7083ff21dbce admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:38.443108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3f65-659d-4c7c-82b8-7083ff21dbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:38.443373 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3f65-659d-4c7c-82b8-7083ff21dbce admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:38.444003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec3f65-659d-4c7c-82b8-7083ff21dbce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:38.445558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec3f65-659d-4c7c-82b8-7083ff21dbce admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:38.456020 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1253/2486] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:48:38.460173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57159f-3746-43a3-9d3b-91018df3bfde admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:38.469987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57159f-3746-43a3-9d3b-91018df3bfde admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:38.475225 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3636db-066c-48ed-b027-0412870a961b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:38.475818 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3636db-066c-48ed-b027-0412870a961b admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:38.476088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3636db-066c-48ed-b027-0412870a961b admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:38.476361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3636db-066c-48ed-b027-0412870a961b admin admin] PATH_INFO: `/v3/projects/86f59bdb0c3a46bf86fe611b95335b19/users/2dcce7be8c974a75a6ac82845865006f/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:38.491672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3636db-066c-48ed-b027-0412870a961b admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f59bdb0c3a46bf86fe611b95335b19, user_id=2dcce7be8c974a75a6ac82845865006f, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:38.494545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3636db-066c-48ed-b027-0412870a961b admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:38.558911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1234/2487] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:48:38 2022] PUT /identity/v3/projects/86f59bdb0c3a46bf86fe611b95335b19/users/2dcce7be8c974a75a6ac82845865006f/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:48:38.564123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473350bb-aba9-4684-812e-3d53fd912948 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:38.566910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473350bb-aba9-4684-812e-3d53fd912948 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:38.566910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473350bb-aba9-4684-812e-3d53fd912948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:38.590143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-473350bb-aba9-4684-812e-3d53fd912948 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:48:38.591403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-473350bb-aba9-4684-812e-3d53fd912948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:38.601872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:48:38.654964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1254/2488] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:48:38.665115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3636db-066c-48ed-b027-0412870a961b admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:38.666856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8b3636db-066c-48ed-b027-0412870a961b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:38.722872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3636db-066c-48ed-b027-0412870a961b admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:38.727137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c65f9-a6a8-4f12-b825-c76fd1c10db2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:38.727672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c65f9-a6a8-4f12-b825-c76fd1c10db2 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:38.727942 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c65f9-a6a8-4f12-b825-c76fd1c10db2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:38.728205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c65f9-a6a8-4f12-b825-c76fd1c10db2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:38.736977 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c65f9-a6a8-4f12-b825-c76fd1c10db2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:38.738421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c65f9-a6a8-4f12-b825-c76fd1c10db2 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:38.748172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1235/2489] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:48:44.572481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56c3d0-91a8-49fe-829a-2193a6db2b08 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:44.573017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56c3d0-91a8-49fe-829a-2193a6db2b08 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:44.573017 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56c3d0-91a8-49fe-829a-2193a6db2b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:44.603399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-3a56c3d0-91a8-49fe-829a-2193a6db2b08 None None] MFA Rules not processed for user `2dcce7be8c974a75a6ac82845865006f`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:48:44.604687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3a56c3d0-91a8-49fe-829a-2193a6db2b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:44.677704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1255/2490] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:48:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:48:44.846527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c65f9-a6a8-4f12-b825-c76fd1c10db2 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:48:44.847909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e58c65f9-a6a8-4f12-b825-c76fd1c10db2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:44.867741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:48:44.947875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c65f9-a6a8-4f12-b825-c76fd1c10db2 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:48:44.953441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc38868-4aa9-4c80-b076-f88fef032776 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:48:44.955186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38868-4aa9-4c80-b076-f88fef032776 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:44.955186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38868-4aa9-4c80-b076-f88fef032776 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:44.955186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38868-4aa9-4c80-b076-f88fef032776 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:44.962835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc38868-4aa9-4c80-b076-f88fef032776 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:48:44.964450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc38868-4aa9-4c80-b076-f88fef032776 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:48:44.982081 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1236/2491] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:48:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:48:47.534933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac544ff-b81c-4c5a-a6e2-122fff66775a None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:48:47.535455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac544ff-b81c-4c5a-a6e2-122fff66775a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:48:47.535455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac544ff-b81c-4c5a-a6e2-122fff66775a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:48:47.557553 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-6ac544ff-b81c-4c5a-a6e2-122fff66775a None None] MFA Rules not processed for user `2dcce7be8c974a75a6ac82845865006f`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:48:47.559599 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-6ac544ff-b81c-4c5a-a6e2-122fff66775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:48:47.566799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1256/2492] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:48:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:48:54.975116 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=31 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:48:57.551344 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=55 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:49:00.258134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c12af4-2a13-43f6-b7a8-3bf844d1fc57 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:00.264913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c12af4-2a13-43f6-b7a8-3bf844d1fc57 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:00.264913 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c12af4-2a13-43f6-b7a8-3bf844d1fc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:00.279010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-85c12af4-2a13-43f6-b7a8-3bf844d1fc57 None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:49:00.280207 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-85c12af4-2a13-43f6-b7a8-3bf844d1fc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:00.338131 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1237/2493] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:49:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:49:00.472954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac544ff-b81c-4c5a-a6e2-122fff66775a None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:00.475106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-6ac544ff-b81c-4c5a-a6e2-122fff66775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:00.571669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac544ff-b81c-4c5a-a6e2-122fff66775a None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:00.582444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab3d525-8c6c-4010-8906-26fe37d65e70 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:00.583090 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3d525-8c6c-4010-8906-26fe37d65e70 service swift] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:00.583372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3d525-8c6c-4010-8906-26fe37d65e70 service swift] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:00.583643 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab3d525-8c6c-4010-8906-26fe37d65e70 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:00.596760 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3d525-8c6c-4010-8906-26fe37d65e70 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:00.598514 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab3d525-8c6c-4010-8906-26fe37d65e70 service swift] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:00.608364 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1257/2494] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:49:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:49:10.330902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=25 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:49:10.622795 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=32 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:49:18.619811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2175139-b445-4eaa-bac8-64536562215a None None] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:18.620377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2175139-b445-4eaa-bac8-64536562215a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:18.620377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2175139-b445-4eaa-bac8-64536562215a None None] PATH_INFO: `/` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:18.621679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1238/2495] 10.222.0.49 () {54 vars in 921 bytes} [Fri Sep 16 05:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 16 05:49:18.626651 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81297674-ec83-40a7-8247-4f9400f04c9c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:18.627099 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81297674-ec83-40a7-8247-4f9400f04c9c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:18.627272 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81297674-ec83-40a7-8247-4f9400f04c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:18.683207 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-81297674-ec83-40a7-8247-4f9400f04c9c None None] MFA Rules not processed for user `7db75c20e9744feb8b5dc78c554f4977`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:49:18.684475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-81297674-ec83-40a7-8247-4f9400f04c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:18.734745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1258/2496] 10.222.0.49 () {58 vars in 1018 bytes} [Fri Sep 16 05:49:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:49:18.746672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2175139-b445-4eaa-bac8-64536562215a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:18.748367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a2175139-b445-4eaa-bac8-64536562215a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:18.807222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2175139-b445-4eaa-bac8-64536562215a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:18.814066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1b9a1-09da-48d2-b8b6-1a20c8e4a965 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d96ec6235843c5b67d87a3d0ff0f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:18.814066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1b9a1-09da-48d2-b8b6-1a20c8e4a965 service glance] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:18.814066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1b9a1-09da-48d2-b8b6-1a20c8e4a965 service glance] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:18.814066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1b9a1-09da-48d2-b8b6-1a20c8e4a965 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:18.834572 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1b9a1-09da-48d2-b8b6-1a20c8e4a965 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:18.835988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1b9a1-09da-48d2-b8b6-1a20c8e4a965 service glance] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:18.846904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1239/2497] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:49:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:49:21.678729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443b29c-278d-44af-8b47-8c35ce35d28a None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:21.679223 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443b29c-278d-44af-8b47-8c35ce35d28a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:21.679330 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443b29c-278d-44af-8b47-8c35ce35d28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:21.699551 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c443b29c-278d-44af-8b47-8c35ce35d28a None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:49:21.700732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c443b29c-278d-44af-8b47-8c35ce35d28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:21.707875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1259/2498] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:49:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:49:21.719839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1b9a1-09da-48d2-b8b6-1a20c8e4a965 service glance] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:21.731268 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1b9a1-09da-48d2-b8b6-1a20c8e4a965 service glance] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:21.736332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fedbf0-0f55-41e6-b337-9a05d6ce0c15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:21.736915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fedbf0-0f55-41e6-b337-9a05d6ce0c15 service swift] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:21.737194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fedbf0-0f55-41e6-b337-9a05d6ce0c15 service swift] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:21.737457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fedbf0-0f55-41e6-b337-9a05d6ce0c15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:21.745999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fedbf0-0f55-41e6-b337-9a05d6ce0c15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:21.747569 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fedbf0-0f55-41e6-b337-9a05d6ce0c15 service swift] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:21.755505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1240/2499] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:49:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:49:31.693225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=36 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:49:31.752222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=32 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:49:39.633174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67430d9d-aec1-4cdd-84cc-27f1c74768e5 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:39.633794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67430d9d-aec1-4cdd-84cc-27f1c74768e5 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:39.633998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67430d9d-aec1-4cdd-84cc-27f1c74768e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:39.653627 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-67430d9d-aec1-4cdd-84cc-27f1c74768e5 None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:49:39.654903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-67430d9d-aec1-4cdd-84cc-27f1c74768e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:39.662618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1260/2500] 10.222.0.49 () {58 vars in 967 bytes} [Fri Sep 16 05:49:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:49:39.676102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fedbf0-0f55-41e6-b337-9a05d6ce0c15 service swift] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:39.687274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fedbf0-0f55-41e6-b337-9a05d6ce0c15 service swift] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:39.692176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f3653-7db8-4840-aa57-0a130053cc29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:39.692766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f3653-7db8-4840-aa57-0a130053cc29 service swift] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:39.693041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f3653-7db8-4840-aa57-0a130053cc29 service swift] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:39.693394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f3653-7db8-4840-aa57-0a130053cc29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:39.701653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f3653-7db8-4840-aa57-0a130053cc29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:39.703240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f3653-7db8-4840-aa57-0a130053cc29 service swift] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:39.711244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1241/2501] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:49:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:49:48.193293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67430d9d-aec1-4cdd-84cc-27f1c74768e5 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:48.212174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67430d9d-aec1-4cdd-84cc-27f1c74768e5 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:48.215952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd6ae2-0c2c-40a4-95aa-11b1c606fa4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:48.216539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd6ae2-0c2c-40a4-95aa-11b1c606fa4a service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:48.216808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd6ae2-0c2c-40a4-95aa-11b1c606fa4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:48.217068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd6ae2-0c2c-40a4-95aa-11b1c606fa4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:48.218776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f8bd6ae2-0c2c-40a4-95aa-11b1c606fa4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:48.252761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd6ae2-0c2c-40a4-95aa-11b1c606fa4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:48.254231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd6ae2-0c2c-40a4-95aa-11b1c606fa4a service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:48.273320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1261/2502] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:49:49.708253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=6 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:49:51.939020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f3653-7db8-4840-aa57-0a130053cc29 service swift] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:51.941107 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e99f3653-7db8-4840-aa57-0a130053cc29 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:52.016795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f3653-7db8-4840-aa57-0a130053cc29 service swift] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:52.023222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca11872-5085-4cac-948e-a679ccaed845 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9975c0dab76f4ccbb4d2107e8e780ae5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:52.025759 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca11872-5085-4cac-948e-a679ccaed845 service placement] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:52.026072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca11872-5085-4cac-948e-a679ccaed845 service placement] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:52.026345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca11872-5085-4cac-948e-a679ccaed845 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:52.029181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-eca11872-5085-4cac-948e-a679ccaed845 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:52.041366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca11872-5085-4cac-948e-a679ccaed845 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:52.043666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca11872-5085-4cac-948e-a679ccaed845 service placement] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:52.054121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1242/2503] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:49:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:49:52.117106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd6ae2-0c2c-40a4-95aa-11b1c606fa4a service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:52.118646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f8bd6ae2-0c2c-40a4-95aa-11b1c606fa4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:52.131929 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd6ae2-0c2c-40a4-95aa-11b1c606fa4a service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:52.136434 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e528614-3c80-4b85-b20f-0a63b412568c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:52.137898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e528614-3c80-4b85-b20f-0a63b412568c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:52.138168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e528614-3c80-4b85-b20f-0a63b412568c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:52.138433 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e528614-3c80-4b85-b20f-0a63b412568c admin admin] PATH_INFO: `/v3/users/edd619bf865645fab0460d3aa847a734` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:52.139540 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e528614-3c80-4b85-b20f-0a63b412568c admin admin] RBAC: Authorizing `identity:delete_user(user_id=edd619bf865645fab0460d3aa847a734)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:52.141442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e528614-3c80-4b85-b20f-0a63b412568c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:52.364076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1e528614-3c80-4b85-b20f-0a63b412568c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edd619bf865645fab0460d3aa847a734'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:52.364734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1e528614-3c80-4b85-b20f-0a63b412568c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edd619bf865645fab0460d3aa847a734'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:52.464326 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1e528614-3c80-4b85-b20f-0a63b412568c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edd619bf865645fab0460d3aa847a734'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:52.465999 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1e528614-3c80-4b85-b20f-0a63b412568c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edd619bf865645fab0460d3aa847a734'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:52.473543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1e528614-3c80-4b85-b20f-0a63b412568c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edd619bf865645fab0460d3aa847a734'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:52.493157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1262/2504] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:49:52 2022] DELETE /identity/v3/users/edd619bf865645fab0460d3aa847a734 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:49:52.747579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca11872-5085-4cac-948e-a679ccaed845 service placement] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:52.759459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca11872-5085-4cac-948e-a679ccaed845 service placement] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:52.818942 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196803b-82b3-4c17-86f5-44596cd925bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:52.819563 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196803b-82b3-4c17-86f5-44596cd925bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:52.819839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196803b-82b3-4c17-86f5-44596cd925bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:52.820106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196803b-82b3-4c17-86f5-44596cd925bc admin admin] PATH_INFO: `/v3/projects/3cf2362acfe94538bba7793bd690649c` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:52.821208 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196803b-82b3-4c17-86f5-44596cd925bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cf2362acfe94538bba7793bd690649c)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:52.823132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196803b-82b3-4c17-86f5-44596cd925bc admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:52.928177 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a196803b-82b3-4c17-86f5-44596cd925bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cf2362acfe94538bba7793bd690649c', 'request_id': 'req-a196803b-82b3-4c17-86f5-44596cd925bc'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:52.936093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a196803b-82b3-4c17-86f5-44596cd925bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cf2362acfe94538bba7793bd690649c', 'request_id': 'req-a196803b-82b3-4c17-86f5-44596cd925bc'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:52.941405 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a196803b-82b3-4c17-86f5-44596cd925bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cf2362acfe94538bba7793bd690649c', 'request_id': 'req-a196803b-82b3-4c17-86f5-44596cd925bc'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:53.023826 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a196803b-82b3-4c17-86f5-44596cd925bc admin admin] The token cache is being invalidate because project 3cf2362acfe94538bba7793bd690649c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:49:53.027950 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a196803b-82b3-4c17-86f5-44596cd925bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:53.027950 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a196803b-82b3-4c17-86f5-44596cd925bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:53.027950 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1243/2505] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:49:52 2022] DELETE /identity/v3/projects/3cf2362acfe94538bba7793bd690649c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:49:53.031436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e50d5-8ca9-4af8-9fdb-72b8da5bde32 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:53.031740 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e50d5-8ca9-4af8-9fdb-72b8da5bde32 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:53.032009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e50d5-8ca9-4af8-9fdb-72b8da5bde32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:53.079009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-df2e50d5-8ca9-4af8-9fdb-72b8da5bde32 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:49:53.080967 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-df2e50d5-8ca9-4af8-9fdb-72b8da5bde32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:53.130788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:49:53.142565 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1263/2506] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:49:53.157781 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b1f29-24a2-4b44-985d-aec93c0aa2da None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:53.158163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b1f29-24a2-4b44-985d-aec93c0aa2da None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:53.158480 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b1f29-24a2-4b44-985d-aec93c0aa2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:53.179132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-a17b1f29-24a2-4b44-985d-aec93c0aa2da None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:49:53.180261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a17b1f29-24a2-4b44-985d-aec93c0aa2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:53.188822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1244/2507] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:49:53.193059 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e50d5-8ca9-4af8-9fdb-72b8da5bde32 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:53.203050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e50d5-8ca9-4af8-9fdb-72b8da5bde32 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:53.207666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fccb05-419f-4271-9554-0e4abc133e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:53.208330 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fccb05-419f-4271-9554-0e4abc133e0c admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:53.208602 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fccb05-419f-4271-9554-0e4abc133e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:53.208868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fccb05-419f-4271-9554-0e4abc133e0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:53.209551 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fccb05-419f-4271-9554-0e4abc133e0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:49:53.209872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fccb05-419f-4271-9554-0e4abc133e0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:53.211449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fccb05-419f-4271-9554-0e4abc133e0c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:53.223874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1264/2508] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:49:53.228374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b1f29-24a2-4b44-985d-aec93c0aa2da None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:53.238158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b1f29-24a2-4b44-985d-aec93c0aa2da None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:53.242822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a6fe1-d556-44c2-97eb-0f5632138d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:53.243415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a6fe1-d556-44c2-97eb-0f5632138d9e admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:53.243688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a6fe1-d556-44c2-97eb-0f5632138d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:53.243955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a6fe1-d556-44c2-97eb-0f5632138d9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:53.244572 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a6fe1-d556-44c2-97eb-0f5632138d9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:53.246641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a6fe1-d556-44c2-97eb-0f5632138d9e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:53.257854 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:49:53.373213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1245/2509] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:49:53 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:49:53.375370 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fccb05-419f-4271-9554-0e4abc133e0c admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:53.386058 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fccb05-419f-4271-9554-0e4abc133e0c admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:53.442887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10674f88-7bdc-41ac-ac42-8c54ee864c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:53.443520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10674f88-7bdc-41ac-ac42-8c54ee864c6c admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:53.443799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10674f88-7bdc-41ac-ac42-8c54ee864c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:53.444067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10674f88-7bdc-41ac-ac42-8c54ee864c6c admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:53.444705 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10674f88-7bdc-41ac-ac42-8c54ee864c6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:53.446643 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10674f88-7bdc-41ac-ac42-8c54ee864c6c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:53.544421 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1265/2510] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:49:53 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:49:53.545645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a6fe1-d556-44c2-97eb-0f5632138d9e admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:53.556364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a6fe1-d556-44c2-97eb-0f5632138d9e admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:53.573379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca6dfe-c32b-4629-9856-56b9448b2c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:53.573379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca6dfe-c32b-4629-9856-56b9448b2c45 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:53.573379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca6dfe-c32b-4629-9856-56b9448b2c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:53.573379 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca6dfe-c32b-4629-9856-56b9448b2c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:53.573754 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca6dfe-c32b-4629-9856-56b9448b2c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:53.573754 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca6dfe-c32b-4629-9856-56b9448b2c45 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:53.576271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1246/2511] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:49:53.580203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10674f88-7bdc-41ac-ac42-8c54ee864c6c admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:53.590437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10674f88-7bdc-41ac-ac42-8c54ee864c6c admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:53.595218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b664c-3aa5-4fac-882f-f20ae0e6cc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:53.595814 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b664c-3aa5-4fac-882f-f20ae0e6cc63 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:53.596090 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b664c-3aa5-4fac-882f-f20ae0e6cc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:53.596366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b664c-3aa5-4fac-882f-f20ae0e6cc63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:53.597000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b664c-3aa5-4fac-882f-f20ae0e6cc63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:53.598632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b664c-3aa5-4fac-882f-f20ae0e6cc63 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:53.608807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1266/2512] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:49:53.612905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca6dfe-c32b-4629-9856-56b9448b2c45 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:53.623984 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca6dfe-c32b-4629-9856-56b9448b2c45 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:53.628790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e9507-7117-4d7c-aa55-d9c148291988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:53.629439 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e9507-7117-4d7c-aa55-d9c148291988 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:53.629715 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e9507-7117-4d7c-aa55-d9c148291988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:53.630048 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e9507-7117-4d7c-aa55-d9c148291988 admin admin] PATH_INFO: `/v3/projects/d986f292fa604393881fc53a45ef667c/users/c92b4c3a573149b291c729f77d70eec7/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:53.646082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e9507-7117-4d7c-aa55-d9c148291988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d986f292fa604393881fc53a45ef667c, user_id=c92b4c3a573149b291c729f77d70eec7, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:53.648722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e9507-7117-4d7c-aa55-d9c148291988 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:53.750556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1247/2513] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:49:53 2022] PUT /identity/v3/projects/d986f292fa604393881fc53a45ef667c/users/c92b4c3a573149b291c729f77d70eec7/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:49:53.760831 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7f820-d12c-4e5e-8c1c-d62c4ada37b5 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:53.761144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7f820-d12c-4e5e-8c1c-d62c4ada37b5 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:53.761397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7f820-d12c-4e5e-8c1c-d62c4ada37b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:53.790824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-e6d7f820-d12c-4e5e-8c1c-d62c4ada37b5 None None] MFA Rules not processed for user `c92b4c3a573149b291c729f77d70eec7`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:49:53.792035 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e6d7f820-d12c-4e5e-8c1c-d62c4ada37b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:53.856910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1267/2514] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:49:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:49:53.868205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e9507-7117-4d7c-aa55-d9c148291988 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:53.869944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-524e9507-7117-4d7c-aa55-d9c148291988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:53.944452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e9507-7117-4d7c-aa55-d9c148291988 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:53.950743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907ee32-7cc9-4166-b885-09023226c325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:53.951342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907ee32-7cc9-4166-b885-09023226c325 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:53.951620 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907ee32-7cc9-4166-b885-09023226c325 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:53.951892 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907ee32-7cc9-4166-b885-09023226c325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:53.959665 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907ee32-7cc9-4166-b885-09023226c325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:53.961253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8907ee32-7cc9-4166-b885-09023226c325 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:53.970725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1248/2515] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:49:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:49:57.173991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7f820-d12c-4e5e-8c1c-d62c4ada37b5 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:57.217547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:49:57.231461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7f820-d12c-4e5e-8c1c-d62c4ada37b5 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:57.235933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:57.239063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:57.239063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:57.239063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] PATH_INFO: `/v3/users/c92b4c3a573149b291c729f77d70eec7` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:57.239063 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c92b4c3a573149b291c729f77d70eec7)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:57.240041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:57.449112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c92b4c3a573149b291c729f77d70eec7'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:57.450749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c92b4c3a573149b291c729f77d70eec7'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:57.515390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c92b4c3a573149b291c729f77d70eec7'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:57.516008 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c92b4c3a573149b291c729f77d70eec7'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:57.523212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c92b4c3a573149b291c729f77d70eec7'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:57.541675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1268/2516] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:49:57 2022] DELETE /identity/v3/users/c92b4c3a573149b291c729f77d70eec7 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:49:57.546474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2a969-b34b-4857-b88b-1ddfde6b8200 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:57.546992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2a969-b34b-4857-b88b-1ddfde6b8200 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:57.546992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2a969-b34b-4857-b88b-1ddfde6b8200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:57.566043 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c9f2a969-b34b-4857-b88b-1ddfde6b8200 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:49:57.567234 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c9f2a969-b34b-4857-b88b-1ddfde6b8200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:57.625516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1249/2517] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:49:57.635156 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:57.637202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:57.705174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e12fb1c-67c3-4d8e-bac0-9661d122dd10 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:57.708761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc2b77-57de-41ab-b4a0-a38550016724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:57.709353 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc2b77-57de-41ab-b4a0-a38550016724 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:57.709628 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc2b77-57de-41ab-b4a0-a38550016724 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:57.709923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc2b77-57de-41ab-b4a0-a38550016724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:57.719822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc2b77-57de-41ab-b4a0-a38550016724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:57.721231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc2b77-57de-41ab-b4a0-a38550016724 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:57.731221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1269/2518] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:49:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:49:58.125320 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2a969-b34b-4857-b88b-1ddfde6b8200 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:58.126912 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c9f2a969-b34b-4857-b88b-1ddfde6b8200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:58.139818 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2a969-b34b-4857-b88b-1ddfde6b8200 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:58.144487 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5685881-c7da-4919-8cbd-6c4f8a17fddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:58.146550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5685881-c7da-4919-8cbd-6c4f8a17fddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:58.146891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5685881-c7da-4919-8cbd-6c4f8a17fddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:58.147253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5685881-c7da-4919-8cbd-6c4f8a17fddc admin admin] PATH_INFO: `/v3/projects/d986f292fa604393881fc53a45ef667c` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:58.148515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5685881-c7da-4919-8cbd-6c4f8a17fddc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d986f292fa604393881fc53a45ef667c)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:58.150553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5685881-c7da-4919-8cbd-6c4f8a17fddc admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:58.250401 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:49:58.265381 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b5685881-c7da-4919-8cbd-6c4f8a17fddc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd986f292fa604393881fc53a45ef667c', 'request_id': 'req-b5685881-c7da-4919-8cbd-6c4f8a17fddc'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:58.270766 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b5685881-c7da-4919-8cbd-6c4f8a17fddc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd986f292fa604393881fc53a45ef667c', 'request_id': 'req-b5685881-c7da-4919-8cbd-6c4f8a17fddc'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:58.276847 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b5685881-c7da-4919-8cbd-6c4f8a17fddc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd986f292fa604393881fc53a45ef667c', 'request_id': 'req-b5685881-c7da-4919-8cbd-6c4f8a17fddc'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:58.396750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b5685881-c7da-4919-8cbd-6c4f8a17fddc admin admin] The token cache is being invalidate because project d986f292fa604393881fc53a45ef667c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:49:58.397049 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b5685881-c7da-4919-8cbd-6c4f8a17fddc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:58.397689 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-b5685881-c7da-4919-8cbd-6c4f8a17fddc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:49:58.399182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1250/2519] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:49:58 2022] DELETE /identity/v3/projects/d986f292fa604393881fc53a45ef667c => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:49:58.404438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f37f71-6e66-458e-93d3-33623fbf3cef None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:58.404924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f37f71-6e66-458e-93d3-33623fbf3cef None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:58.405019 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f37f71-6e66-458e-93d3-33623fbf3cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:58.425423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-28f37f71-6e66-458e-93d3-33623fbf3cef None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:49:58.426641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-28f37f71-6e66-458e-93d3-33623fbf3cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:58.477899 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:49:58.486596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1270/2520] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:49:58.494599 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0267e0b-d31a-4c3b-9401-e65da0efa7a0 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:58.494903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0267e0b-d31a-4c3b-9401-e65da0efa7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:58.495173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0267e0b-d31a-4c3b-9401-e65da0efa7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:58.515838 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-a0267e0b-d31a-4c3b-9401-e65da0efa7a0 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:49:58.516963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a0267e0b-d31a-4c3b-9401-e65da0efa7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:58.524771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1251/2521] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:49:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:49:58.529756 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f37f71-6e66-458e-93d3-33623fbf3cef None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:58.539764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f37f71-6e66-458e-93d3-33623fbf3cef None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:58.545147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c497fd-017a-4215-8a06-9109c6c9c71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:58.545745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c497fd-017a-4215-8a06-9109c6c9c71e admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:58.546047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c497fd-017a-4215-8a06-9109c6c9c71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:58.546319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c497fd-017a-4215-8a06-9109c6c9c71e admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:58.546980 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c497fd-017a-4215-8a06-9109c6c9c71e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:49:58.547276 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c497fd-017a-4215-8a06-9109c6c9c71e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:58.548856 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c497fd-017a-4215-8a06-9109c6c9c71e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:58.560844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1271/2522] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:49:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:49:58.565134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0267e0b-d31a-4c3b-9401-e65da0efa7a0 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:58.574564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0267e0b-d31a-4c3b-9401-e65da0efa7a0 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:58.579279 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990aacf-fb32-4cc2-8441-fd66f06b4968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:58.579870 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990aacf-fb32-4cc2-8441-fd66f06b4968 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:58.580147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990aacf-fb32-4cc2-8441-fd66f06b4968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:58.580415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990aacf-fb32-4cc2-8441-fd66f06b4968 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:58.581050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990aacf-fb32-4cc2-8441-fd66f06b4968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:58.582957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990aacf-fb32-4cc2-8441-fd66f06b4968 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:58.667765 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1252/2523] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:49:58 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:49:58.672429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c497fd-017a-4215-8a06-9109c6c9c71e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:58.682399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c497fd-017a-4215-8a06-9109c6c9c71e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:58.737537 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524a3c0-a345-4be7-b658-4991ffeef962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:58.738166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524a3c0-a345-4be7-b658-4991ffeef962 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:58.738444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524a3c0-a345-4be7-b658-4991ffeef962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:58.738713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524a3c0-a345-4be7-b658-4991ffeef962 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:58.739346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524a3c0-a345-4be7-b658-4991ffeef962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:58.741250 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524a3c0-a345-4be7-b658-4991ffeef962 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:58.838776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1272/2524] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:49:58 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:49:58.844233 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990aacf-fb32-4cc2-8441-fd66f06b4968 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:58.856458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990aacf-fb32-4cc2-8441-fd66f06b4968 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:58.861100 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1f439-2aa4-4c65-9260-ea09c19c6619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:58.861799 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1f439-2aa4-4c65-9260-ea09c19c6619 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:58.862101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1f439-2aa4-4c65-9260-ea09c19c6619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:58.863013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb1f439-2aa4-4c65-9260-ea09c19c6619 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:58.863182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1f439-2aa4-4c65-9260-ea09c19c6619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:58.864909 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb1f439-2aa4-4c65-9260-ea09c19c6619 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:58.876281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1253/2525] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:49:58.880209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524a3c0-a345-4be7-b658-4991ffeef962 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:58.890466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524a3c0-a345-4be7-b658-4991ffeef962 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:58.895251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbd055-387b-49a2-9db6-ea9231997bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:58.895887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbd055-387b-49a2-9db6-ea9231997bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:58.896193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbd055-387b-49a2-9db6-ea9231997bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:58.896464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbd055-387b-49a2-9db6-ea9231997bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:58.898865 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbd055-387b-49a2-9db6-ea9231997bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:58.898865 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbd055-387b-49a2-9db6-ea9231997bd7 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:58.910471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1273/2526] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:49:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:49:58.914811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1f439-2aa4-4c65-9260-ea09c19c6619 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:58.926627 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb1f439-2aa4-4c65-9260-ea09c19c6619 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:58.931400 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ee1a5-6212-4514-9b02-469b0c950646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:58.932059 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ee1a5-6212-4514-9b02-469b0c950646 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:58.932327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ee1a5-6212-4514-9b02-469b0c950646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:58.932592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ee1a5-6212-4514-9b02-469b0c950646 admin admin] PATH_INFO: `/v3/projects/605065163c084ad0a3ecdd27add44d44/users/516acfddaf964161a7e0086ffc6e3af7/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:58.947178 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ee1a5-6212-4514-9b02-469b0c950646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=605065163c084ad0a3ecdd27add44d44, user_id=516acfddaf964161a7e0086ffc6e3af7, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:58.950404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ee1a5-6212-4514-9b02-469b0c950646 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:59.018965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1254/2527] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:49:58 2022] PUT /identity/v3/projects/605065163c084ad0a3ecdd27add44d44/users/516acfddaf964161a7e0086ffc6e3af7/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:49:59.038848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b7ebb-24b1-4b41-975a-51fdc2d2b66f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:59.039276 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b7ebb-24b1-4b41-975a-51fdc2d2b66f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:59.039544 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b7ebb-24b1-4b41-975a-51fdc2d2b66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:59.067828 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-ad8b7ebb-24b1-4b41-975a-51fdc2d2b66f None None] MFA Rules not processed for user `516acfddaf964161a7e0086ffc6e3af7`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:49:59.069066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ad8b7ebb-24b1-4b41-975a-51fdc2d2b66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:59.133056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1274/2528] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:49:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:49:59.138247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ee1a5-6212-4514-9b02-469b0c950646 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:59.198737 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ee1a5-6212-4514-9b02-469b0c950646 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:59.203880 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab9742f-7914-4365-92fe-dd623e841435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:59.204469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9742f-7914-4365-92fe-dd623e841435 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:59.204738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9742f-7914-4365-92fe-dd623e841435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:59.204998 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab9742f-7914-4365-92fe-dd623e841435 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:59.205619 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab9742f-7914-4365-92fe-dd623e841435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:59.207522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab9742f-7914-4365-92fe-dd623e841435 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:59.285388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1255/2529] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:49:59 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:49:59.289958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b7ebb-24b1-4b41-975a-51fdc2d2b66f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:59.300718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b7ebb-24b1-4b41-975a-51fdc2d2b66f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:59.352160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:49:59.359209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e0a72-88ed-4d11-bdef-ca5e47aa5ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:59.359822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e0a72-88ed-4d11-bdef-ca5e47aa5ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:59.360102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e0a72-88ed-4d11-bdef-ca5e47aa5ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:59.360379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e0a72-88ed-4d11-bdef-ca5e47aa5ac7 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:59.361008 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e0a72-88ed-4d11-bdef-ca5e47aa5ac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:59.363009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e0a72-88ed-4d11-bdef-ca5e47aa5ac7 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:59.464512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1275/2530] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:49:59 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:49:59.465681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab9742f-7914-4365-92fe-dd623e841435 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:59.476158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab9742f-7914-4365-92fe-dd623e841435 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:59.481096 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc2833-d7cc-4835-8671-dd845151f09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:59.481692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc2833-d7cc-4835-8671-dd845151f09a admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:59.482013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc2833-d7cc-4835-8671-dd845151f09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:59.482290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc2833-d7cc-4835-8671-dd845151f09a admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:59.482928 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc2833-d7cc-4835-8671-dd845151f09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:59.484515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc2833-d7cc-4835-8671-dd845151f09a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:59.495565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1256/2531] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:49:59.500488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e0a72-88ed-4d11-bdef-ca5e47aa5ac7 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:59.510520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e0a72-88ed-4d11-bdef-ca5e47aa5ac7 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:59.516566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3bb16-8cca-4ecd-bb68-765b961439db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:59.521995 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3bb16-8cca-4ecd-bb68-765b961439db admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:59.521995 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3bb16-8cca-4ecd-bb68-765b961439db admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:59.521995 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3bb16-8cca-4ecd-bb68-765b961439db admin admin] PATH_INFO: `/v3/projects/9ca438e0dee94f658d21cf1070aca34c/users/d43dc6d9de17458da3ff612e1293f271/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:59.532757 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3bb16-8cca-4ecd-bb68-765b961439db admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca438e0dee94f658d21cf1070aca34c, user_id=d43dc6d9de17458da3ff612e1293f271, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:59.535556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3bb16-8cca-4ecd-bb68-765b961439db admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:59.597535 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1276/2532] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:49:59 2022] PUT /identity/v3/projects/9ca438e0dee94f658d21cf1070aca34c/users/d43dc6d9de17458da3ff612e1293f271/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:49:59.601241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc2833-d7cc-4835-8671-dd845151f09a admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:59.612003 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc2833-d7cc-4835-8671-dd845151f09a admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:59.638195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:49:59.671275 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf0ee9-5e2b-4b29-8b17-9e07c2546e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:59.671869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf0ee9-5e2b-4b29-8b17-9e07c2546e5c admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:59.672143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf0ee9-5e2b-4b29-8b17-9e07c2546e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:59.672407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf0ee9-5e2b-4b29-8b17-9e07c2546e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:59.673033 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf0ee9-5e2b-4b29-8b17-9e07c2546e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:59.674690 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf0ee9-5e2b-4b29-8b17-9e07c2546e5c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:59.684359 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1257/2533] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:49:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:49:59.689293 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3bb16-8cca-4ecd-bb68-765b961439db admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:59.699807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3bb16-8cca-4ecd-bb68-765b961439db admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:59.704882 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29495f39-9054-408d-ab88-9920b9d57856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:59.705646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29495f39-9054-408d-ab88-9920b9d57856 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:59.705789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29495f39-9054-408d-ab88-9920b9d57856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:59.706108 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29495f39-9054-408d-ab88-9920b9d57856 admin admin] PATH_INFO: `/v3/projects/9ca438e0dee94f658d21cf1070aca34c/users/d43dc6d9de17458da3ff612e1293f271/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:59.708044 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29495f39-9054-408d-ab88-9920b9d57856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ca438e0dee94f658d21cf1070aca34c, user_id=d43dc6d9de17458da3ff612e1293f271, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:59.710693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29495f39-9054-408d-ab88-9920b9d57856 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:59.722180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1277/2534] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:49:59 2022] PUT /identity/v3/projects/9ca438e0dee94f658d21cf1070aca34c/users/d43dc6d9de17458da3ff612e1293f271/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:49:59.733298 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea52e0-be42-4f6c-83fc-cb2867b04608 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:59.733749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea52e0-be42-4f6c-83fc-cb2867b04608 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:59.734061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea52e0-be42-4f6c-83fc-cb2867b04608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:59.764430 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-61ea52e0-be42-4f6c-83fc-cb2867b04608 None None] MFA Rules not processed for user `d43dc6d9de17458da3ff612e1293f271`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:49:59.765682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-61ea52e0-be42-4f6c-83fc-cb2867b04608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:59.832899 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1258/2535] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:49:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:49:59.845319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29495f39-9054-408d-ab88-9920b9d57856 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:49:59.846979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-29495f39-9054-408d-ab88-9920b9d57856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:49:59.934836 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29495f39-9054-408d-ab88-9920b9d57856 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:49:59.938594 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc9a3e-a26e-4393-a284-4af54f6c46d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:49:59.939197 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc9a3e-a26e-4393-a284-4af54f6c46d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:49:59.939525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc9a3e-a26e-4393-a284-4af54f6c46d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:49:59.939741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc9a3e-a26e-4393-a284-4af54f6c46d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:49:59.949141 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc9a3e-a26e-4393-a284-4af54f6c46d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:49:59.952182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc9a3e-a26e-4393-a284-4af54f6c46d4 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:49:59.961998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1278/2536] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:49:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:01.191612 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea52e0-be42-4f6c-83fc-cb2867b04608 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:01.200706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea52e0-be42-4f6c-83fc-cb2867b04608 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:01.205422 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266ff75-603b-44c0-8644-2163240316cd tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd43dc6d9de17458da3ff612e1293f271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca438e0dee94f658d21cf1070aca34c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:01.206140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266ff75-603b-44c0-8644-2163240316cd tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:01.206468 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266ff75-603b-44c0-8644-2163240316cd tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:01.206807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266ff75-603b-44c0-8644-2163240316cd tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:01.207516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266ff75-603b-44c0-8644-2163240316cd tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:50:01.207823 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266ff75-603b-44c0-8644-2163240316cd tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:01.209408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266ff75-603b-44c0-8644-2163240316cd tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:01.223547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1259/2537] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:50:01.228683 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc9a3e-a26e-4393-a284-4af54f6c46d4 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:01.238971 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc9a3e-a26e-4393-a284-4af54f6c46d4 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:01.243641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd6a96-bd2c-4b6b-b195-aca71858a368 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd43dc6d9de17458da3ff612e1293f271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca438e0dee94f658d21cf1070aca34c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:01.244260 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd6a96-bd2c-4b6b-b195-aca71858a368 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:01.244532 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd6a96-bd2c-4b6b-b195-aca71858a368 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:01.244798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd6a96-bd2c-4b6b-b195-aca71858a368 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:01.245523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd6a96-bd2c-4b6b-b195-aca71858a368 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:01.247424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd6a96-bd2c-4b6b-b195-aca71858a368 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:01.337522 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1279/2538] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:01 2022] POST /identity/v3/projects => generated 354 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:01.339320 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266ff75-603b-44c0-8644-2163240316cd tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:01.350579 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266ff75-603b-44c0-8644-2163240316cd tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:01.414871 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee7f26-13e2-49ef-b517-509a04134b76 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd43dc6d9de17458da3ff612e1293f271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca438e0dee94f658d21cf1070aca34c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:01.415797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee7f26-13e2-49ef-b517-509a04134b76 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:01.416070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee7f26-13e2-49ef-b517-509a04134b76 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:01.416339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ee7f26-13e2-49ef-b517-509a04134b76 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:01.416976 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee7f26-13e2-49ef-b517-509a04134b76 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:50:01.417280 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee7f26-13e2-49ef-b517-509a04134b76 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:01.418991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ee7f26-13e2-49ef-b517-509a04134b76 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:01.432631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1260/2539] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:50:01.470939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd6a96-bd2c-4b6b-b195-aca71858a368 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:01.481027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd6a96-bd2c-4b6b-b195-aca71858a368 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:01.486569 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd9c0c-f527-4c01-81ee-448c1740703d tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd43dc6d9de17458da3ff612e1293f271', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9ca438e0dee94f658d21cf1070aca34c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:01.487190 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd9c0c-f527-4c01-81ee-448c1740703d tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:01.487483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd9c0c-f527-4c01-81ee-448c1740703d tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:01.487750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd9c0c-f527-4c01-81ee-448c1740703d tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] PATH_INFO: `/v3/projects/0882899fc6d5467898a3f453fb56f841` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:01.488855 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd9c0c-f527-4c01-81ee-448c1740703d tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0882899fc6d5467898a3f453fb56f841)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:01.490788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd9c0c-f527-4c01-81ee-448c1740703d tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:01.621875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-29dd9c0c-f527-4c01-81ee-448c1740703d tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0882899fc6d5467898a3f453fb56f841', 'request_id': 'req-29dd9c0c-f527-4c01-81ee-448c1740703d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:01.627656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-29dd9c0c-f527-4c01-81ee-448c1740703d tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0882899fc6d5467898a3f453fb56f841', 'request_id': 'req-29dd9c0c-f527-4c01-81ee-448c1740703d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:01.634737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-29dd9c0c-f527-4c01-81ee-448c1740703d tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0882899fc6d5467898a3f453fb56f841', 'request_id': 'req-29dd9c0c-f527-4c01-81ee-448c1740703d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:01.636508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:01.723904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-29dd9c0c-f527-4c01-81ee-448c1740703d tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] The token cache is being invalidate because project 0882899fc6d5467898a3f453fb56f841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:01.724198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-29dd9c0c-f527-4c01-81ee-448c1740703d tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:01.726978 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-29dd9c0c-f527-4c01-81ee-448c1740703d tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:01.726978 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1280/2540] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:01 2022] DELETE /identity/v3/projects/0882899fc6d5467898a3f453fb56f841 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:02.461520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee7f26-13e2-49ef-b517-509a04134b76 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:02.463624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-05ee7f26-13e2-49ef-b517-509a04134b76 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:02.494507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:02.527564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ee7f26-13e2-49ef-b517-509a04134b76 tempest-VolumeQuotaClassesTest-1009597633 tempest-VolumeQuotaClassesTest-1009597633-project-admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:02.566088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c3788-5fbd-4bed-9ca1-04bafcd2fcb7 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:02.566641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c3788-5fbd-4bed-9ca1-04bafcd2fcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:02.566641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c3788-5fbd-4bed-9ca1-04bafcd2fcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:02.585474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-388c3788-5fbd-4bed-9ca1-04bafcd2fcb7 None None] MFA Rules not processed for user `2dcce7be8c974a75a6ac82845865006f`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:02.586417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:02.586712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-388c3788-5fbd-4bed-9ca1-04bafcd2fcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:02.587047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:02.587320 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:02.587587 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] PATH_INFO: `/v3/users/516acfddaf964161a7e0086ffc6e3af7` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:02.588719 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=516acfddaf964161a7e0086ffc6e3af7)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:02.590702 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:02.644212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1281/2541] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:50:02.730014 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '516acfddaf964161a7e0086ffc6e3af7'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:02.730939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '516acfddaf964161a7e0086ffc6e3af7'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:02.818500 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '516acfddaf964161a7e0086ffc6e3af7'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:02.826794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '516acfddaf964161a7e0086ffc6e3af7'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:02.827681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '516acfddaf964161a7e0086ffc6e3af7'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:02.847161 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1261/2542] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:02 2022] DELETE /identity/v3/users/516acfddaf964161a7e0086ffc6e3af7 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:50:02.851678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d18ac-036d-45a1-98a6-691ea2371367 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:02.851996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d18ac-036d-45a1-98a6-691ea2371367 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:02.852254 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d18ac-036d-45a1-98a6-691ea2371367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:02.872111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-0a6d18ac-036d-45a1-98a6-691ea2371367 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:02.874261 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0a6d18ac-036d-45a1-98a6-691ea2371367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:02.949564 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1282/2543] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:02.957563 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:02.959143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:03.025246 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0a43d-d90d-4102-84c6-c9a45a2578e2 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:03.030015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fd24f-13b2-461d-939b-af0150cdcca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:03.030742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd24f-13b2-461d-939b-af0150cdcca4 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:03.031079 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd24f-13b2-461d-939b-af0150cdcca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:03.031404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fd24f-13b2-461d-939b-af0150cdcca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:03.045389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fd24f-13b2-461d-939b-af0150cdcca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:03.047187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fd24f-13b2-461d-939b-af0150cdcca4 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:03.051509 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:03.058365 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1262/2544] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:03.610449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d18ac-036d-45a1-98a6-691ea2371367 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:03.611879 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0a6d18ac-036d-45a1-98a6-691ea2371367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:03.624042 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d18ac-036d-45a1-98a6-691ea2371367 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:03.628956 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:03.629649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:03.636032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:03.636032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] PATH_INFO: `/v3/projects/605065163c084ad0a3ecdd27add44d44` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:03.636032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=605065163c084ad0a3ecdd27add44d44)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:03.636032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:03.768804 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '605065163c084ad0a3ecdd27add44d44', 'request_id': 'req-5129e87e-155b-4aea-ae76-5a9813e93c4f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:03.775422 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '605065163c084ad0a3ecdd27add44d44', 'request_id': 'req-5129e87e-155b-4aea-ae76-5a9813e93c4f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:03.783491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '605065163c084ad0a3ecdd27add44d44', 'request_id': 'req-5129e87e-155b-4aea-ae76-5a9813e93c4f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:03.869647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] The token cache is being invalidate because project 605065163c084ad0a3ecdd27add44d44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:03.870032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:03.870704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:03.872175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1283/2545] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:03 2022] DELETE /identity/v3/projects/605065163c084ad0a3ecdd27add44d44 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:03.876895 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fd24f-13b2-461d-939b-af0150cdcca4 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:03.886836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fd24f-13b2-461d-939b-af0150cdcca4 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:03.944017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07545e8f-9b38-4794-83f0-bf843ad42b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:03.944644 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07545e8f-9b38-4794-83f0-bf843ad42b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:03.944917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07545e8f-9b38-4794-83f0-bf843ad42b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:03.945183 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07545e8f-9b38-4794-83f0-bf843ad42b61 admin admin] PATH_INFO: `/v3/users/d43dc6d9de17458da3ff612e1293f271` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:03.946309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07545e8f-9b38-4794-83f0-bf843ad42b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d43dc6d9de17458da3ff612e1293f271)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:03.948317 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07545e8f-9b38-4794-83f0-bf843ad42b61 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:04.135846 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-07545e8f-9b38-4794-83f0-bf843ad42b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd43dc6d9de17458da3ff612e1293f271'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:04.136494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-07545e8f-9b38-4794-83f0-bf843ad42b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd43dc6d9de17458da3ff612e1293f271'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:04.241482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-07545e8f-9b38-4794-83f0-bf843ad42b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd43dc6d9de17458da3ff612e1293f271'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:04.250186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-07545e8f-9b38-4794-83f0-bf843ad42b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd43dc6d9de17458da3ff612e1293f271'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:04.250856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-07545e8f-9b38-4794-83f0-bf843ad42b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd43dc6d9de17458da3ff612e1293f271'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:04.269926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1263/2546] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:03 2022] DELETE /identity/v3/users/d43dc6d9de17458da3ff612e1293f271 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:04.802240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:04.802240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:04.814867 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129e87e-155b-4aea-ae76-5a9813e93c4f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:04.840605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:04.884330 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf4a99-39ca-4cca-8724-0a0135d0c185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:04.884963 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf4a99-39ca-4cca-8724-0a0135d0c185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:04.887132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf4a99-39ca-4cca-8724-0a0135d0c185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:04.887132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf4a99-39ca-4cca-8724-0a0135d0c185 admin admin] PATH_INFO: `/v3/projects/9ca438e0dee94f658d21cf1070aca34c` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:04.887132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf4a99-39ca-4cca-8724-0a0135d0c185 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ca438e0dee94f658d21cf1070aca34c)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:04.889105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf4a99-39ca-4cca-8724-0a0135d0c185 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:05.033325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-76cf4a99-39ca-4cca-8724-0a0135d0c185 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ca438e0dee94f658d21cf1070aca34c', 'request_id': 'req-76cf4a99-39ca-4cca-8724-0a0135d0c185'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:05.039003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-76cf4a99-39ca-4cca-8724-0a0135d0c185 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ca438e0dee94f658d21cf1070aca34c', 'request_id': 'req-76cf4a99-39ca-4cca-8724-0a0135d0c185'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:05.045754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-76cf4a99-39ca-4cca-8724-0a0135d0c185 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ca438e0dee94f658d21cf1070aca34c', 'request_id': 'req-76cf4a99-39ca-4cca-8724-0a0135d0c185'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:05.129485 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-76cf4a99-39ca-4cca-8724-0a0135d0c185 admin admin] The token cache is being invalidate because project 9ca438e0dee94f658d21cf1070aca34c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:05.132686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-76cf4a99-39ca-4cca-8724-0a0135d0c185 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:05.132686 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-76cf4a99-39ca-4cca-8724-0a0135d0c185 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:05.137493 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1284/2547] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:04 2022] DELETE /identity/v3/projects/9ca438e0dee94f658d21cf1070aca34c => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:05.143127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fc3eb-501e-490e-803c-2ba5faf7fb25 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:05.143630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fc3eb-501e-490e-803c-2ba5faf7fb25 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:05.143757 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fc3eb-501e-490e-803c-2ba5faf7fb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:05.163166 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0e7fc3eb-501e-490e-803c-2ba5faf7fb25 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:05.166029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0e7fc3eb-501e-490e-803c-2ba5faf7fb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:05.227990 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1264/2548] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:05.236853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6deb70-2ced-4ad4-ab6b-b5f82e9c2683 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:05.237169 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6deb70-2ced-4ad4-ab6b-b5f82e9c2683 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:05.237438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6deb70-2ced-4ad4-ab6b-b5f82e9c2683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:05.258768 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-cc6deb70-2ced-4ad4-ab6b-b5f82e9c2683 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:05.259962 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-cc6deb70-2ced-4ad4-ab6b-b5f82e9c2683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:05.268025 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1285/2549] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:05.272267 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fc3eb-501e-490e-803c-2ba5faf7fb25 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:05.282786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7fc3eb-501e-490e-803c-2ba5faf7fb25 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:05.287926 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a6ed6-19b6-49dc-bee4-22424e74254e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:05.288522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a6ed6-19b6-49dc-bee4-22424e74254e admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:05.288794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a6ed6-19b6-49dc-bee4-22424e74254e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:05.289060 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a6ed6-19b6-49dc-bee4-22424e74254e admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:05.289720 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a6ed6-19b6-49dc-bee4-22424e74254e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:50:05.290058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a6ed6-19b6-49dc-bee4-22424e74254e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:05.291638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a6ed6-19b6-49dc-bee4-22424e74254e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:05.305076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1265/2550] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:50:05.313591 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6deb70-2ced-4ad4-ab6b-b5f82e9c2683 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:05.324043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6deb70-2ced-4ad4-ab6b-b5f82e9c2683 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:05.329244 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b4a8d-a9c7-47b2-9ee1-43326ca76bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:05.329869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b4a8d-a9c7-47b2-9ee1-43326ca76bff admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:05.330148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b4a8d-a9c7-47b2-9ee1-43326ca76bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:05.330420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8b4a8d-a9c7-47b2-9ee1-43326ca76bff admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:05.331047 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b4a8d-a9c7-47b2-9ee1-43326ca76bff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:05.332949 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8b4a8d-a9c7-47b2-9ee1-43326ca76bff admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:05.444380 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1286/2551] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:05 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:05.448847 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a6ed6-19b6-49dc-bee4-22424e74254e admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:05.455262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:05.458935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a6ed6-19b6-49dc-bee4-22424e74254e admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:05.514904 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea7d2e-fa48-48b1-8118-99edd733eda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:05.515500 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea7d2e-fa48-48b1-8118-99edd733eda4 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:05.515771 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea7d2e-fa48-48b1-8118-99edd733eda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:05.516033 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea7d2e-fa48-48b1-8118-99edd733eda4 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:05.516706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea7d2e-fa48-48b1-8118-99edd733eda4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:05.518631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea7d2e-fa48-48b1-8118-99edd733eda4 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:05.629978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1266/2552] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:05 2022] POST /identity/v3/users => generated 291 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:05.638051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b4a8d-a9c7-47b2-9ee1-43326ca76bff admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:05.645536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8b4a8d-a9c7-47b2-9ee1-43326ca76bff admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:05.650211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e336bb-a6d9-4113-aa72-7bc66e9b6794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:05.650800 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e336bb-a6d9-4113-aa72-7bc66e9b6794 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:05.651127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e336bb-a6d9-4113-aa72-7bc66e9b6794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:05.651412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e336bb-a6d9-4113-aa72-7bc66e9b6794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:05.652045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e336bb-a6d9-4113-aa72-7bc66e9b6794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:05.653609 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e336bb-a6d9-4113-aa72-7bc66e9b6794 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:05.666448 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1287/2553] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:05.670428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea7d2e-fa48-48b1-8118-99edd733eda4 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:05.680704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea7d2e-fa48-48b1-8118-99edd733eda4 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:05.685429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7053cde-f807-4876-94db-7aee2e8920ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:05.686190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7053cde-f807-4876-94db-7aee2e8920ce admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:05.686470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7053cde-f807-4876-94db-7aee2e8920ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:05.686740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7053cde-f807-4876-94db-7aee2e8920ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:05.687375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7053cde-f807-4876-94db-7aee2e8920ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:05.688948 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7053cde-f807-4876-94db-7aee2e8920ce admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:05.699443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1267/2554] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:50:05.704269 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e336bb-a6d9-4113-aa72-7bc66e9b6794 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:05.714137 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e336bb-a6d9-4113-aa72-7bc66e9b6794 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:05.719465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1d621-7f5e-4bff-ba19-8c8cfa863a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:05.720111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1d621-7f5e-4bff-ba19-8c8cfa863a35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:05.720383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1d621-7f5e-4bff-ba19-8c8cfa863a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:05.720649 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1d621-7f5e-4bff-ba19-8c8cfa863a35 admin admin] PATH_INFO: `/v3/projects/2c984014cb4044aa8fae1cca1553f7af/users/519b8e2496c8422ebe717b3d9940d9e8/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:05.735182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1d621-7f5e-4bff-ba19-8c8cfa863a35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c984014cb4044aa8fae1cca1553f7af, user_id=519b8e2496c8422ebe717b3d9940d9e8, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:05.737848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1d621-7f5e-4bff-ba19-8c8cfa863a35 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:05.818503 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1288/2555] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:05 2022] PUT /identity/v3/projects/2c984014cb4044aa8fae1cca1553f7af/users/519b8e2496c8422ebe717b3d9940d9e8/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:05.830558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444ec98-f0e2-46fd-9859-c08caff9c9e9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:05.830839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444ec98-f0e2-46fd-9859-c08caff9c9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:05.831110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444ec98-f0e2-46fd-9859-c08caff9c9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:05.859964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-e444ec98-f0e2-46fd-9859-c08caff9c9e9 None None] MFA Rules not processed for user `519b8e2496c8422ebe717b3d9940d9e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:05.861171 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e444ec98-f0e2-46fd-9859-c08caff9c9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:05.942547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1268/2556] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:05.947089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1d621-7f5e-4bff-ba19-8c8cfa863a35 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:06.005469 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:06.028347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1d621-7f5e-4bff-ba19-8c8cfa863a35 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:06.034232 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c8812-4acb-414c-b462-f93d0cacb6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:06.034833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c8812-4acb-414c-b462-f93d0cacb6dc admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:06.035267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c8812-4acb-414c-b462-f93d0cacb6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:06.035533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c8812-4acb-414c-b462-f93d0cacb6dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:06.036523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c8812-4acb-414c-b462-f93d0cacb6dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:06.038566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c8812-4acb-414c-b462-f93d0cacb6dc admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:06.123200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1289/2557] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:05 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:06.127313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444ec98-f0e2-46fd-9859-c08caff9c9e9 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:06.138469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444ec98-f0e2-46fd-9859-c08caff9c9e9 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:06.213119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114d9e7-68fc-49c5-a2fc-2df5ffac45fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:06.213787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114d9e7-68fc-49c5-a2fc-2df5ffac45fb admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:06.214008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114d9e7-68fc-49c5-a2fc-2df5ffac45fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:06.214392 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114d9e7-68fc-49c5-a2fc-2df5ffac45fb admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:06.215054 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114d9e7-68fc-49c5-a2fc-2df5ffac45fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:06.217735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114d9e7-68fc-49c5-a2fc-2df5ffac45fb admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:06.311715 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1269/2558] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:06 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:06.312536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c8812-4acb-414c-b462-f93d0cacb6dc admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:06.323974 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c8812-4acb-414c-b462-f93d0cacb6dc admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:06.331446 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904c737-d13d-4284-9bd0-4689867755ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:06.332154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904c737-d13d-4284-9bd0-4689867755ea admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:06.332503 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904c737-d13d-4284-9bd0-4689867755ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:06.332840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904c737-d13d-4284-9bd0-4689867755ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:06.333555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904c737-d13d-4284-9bd0-4689867755ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:06.335289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1904c737-d13d-4284-9bd0-4689867755ea admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:06.347576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1290/2559] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:06.351953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114d9e7-68fc-49c5-a2fc-2df5ffac45fb admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:06.358958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:06.364584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114d9e7-68fc-49c5-a2fc-2df5ffac45fb admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:06.371951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a623500-b15b-4c28-85b5-8861bd2533f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:06.372861 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a623500-b15b-4c28-85b5-8861bd2533f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:06.377071 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a623500-b15b-4c28-85b5-8861bd2533f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:06.377071 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a623500-b15b-4c28-85b5-8861bd2533f5 admin admin] PATH_INFO: `/v3/projects/f817d9776f1740c88b766511ab6ba7fc/users/b84aa7c3011b4238bb3ae00d712936ec/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:06.391741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a623500-b15b-4c28-85b5-8861bd2533f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f817d9776f1740c88b766511ab6ba7fc, user_id=b84aa7c3011b4238bb3ae00d712936ec, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:06.394452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a623500-b15b-4c28-85b5-8861bd2533f5 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:06.527827 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1270/2560] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:06 2022] PUT /identity/v3/projects/f817d9776f1740c88b766511ab6ba7fc/users/b84aa7c3011b4238bb3ae00d712936ec/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:06.532543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904c737-d13d-4284-9bd0-4689867755ea admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:06.543598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904c737-d13d-4284-9bd0-4689867755ea admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:06.602217 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280273a-245f-40b0-a84c-aceb4ddc039a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:06.603119 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280273a-245f-40b0-a84c-aceb4ddc039a admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:06.603403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280273a-245f-40b0-a84c-aceb4ddc039a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:06.603675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280273a-245f-40b0-a84c-aceb4ddc039a admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:06.604319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280273a-245f-40b0-a84c-aceb4ddc039a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:06.605938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280273a-245f-40b0-a84c-aceb4ddc039a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:06.617235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1291/2561] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:06.622794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a623500-b15b-4c28-85b5-8861bd2533f5 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:06.631957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a623500-b15b-4c28-85b5-8861bd2533f5 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:06.636958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9ade6-4448-4a4c-aa65-5f51b8d0791e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:06.637621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9ade6-4448-4a4c-aa65-5f51b8d0791e admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:06.637929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9ade6-4448-4a4c-aa65-5f51b8d0791e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:06.639117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c9ade6-4448-4a4c-aa65-5f51b8d0791e admin admin] PATH_INFO: `/v3/projects/f817d9776f1740c88b766511ab6ba7fc/users/b84aa7c3011b4238bb3ae00d712936ec/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:06.640965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9ade6-4448-4a4c-aa65-5f51b8d0791e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f817d9776f1740c88b766511ab6ba7fc, user_id=b84aa7c3011b4238bb3ae00d712936ec, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:06.643845 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c9ade6-4448-4a4c-aa65-5f51b8d0791e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:06.654436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1271/2562] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:06 2022] PUT /identity/v3/projects/f817d9776f1740c88b766511ab6ba7fc/users/b84aa7c3011b4238bb3ae00d712936ec/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:50:06.665245 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078238b-b88a-4a91-a291-29235abdf40e None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:06.665672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078238b-b88a-4a91-a291-29235abdf40e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:06.666049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078238b-b88a-4a91-a291-29235abdf40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:06.695533 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a078238b-b88a-4a91-a291-29235abdf40e None None] MFA Rules not processed for user `b84aa7c3011b4238bb3ae00d712936ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:06.696752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a078238b-b88a-4a91-a291-29235abdf40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:06.766478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1292/2563] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:06.777232 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9ade6-4448-4a4c-aa65-5f51b8d0791e admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:06.779509 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-10c9ade6-4448-4a4c-aa65-5f51b8d0791e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:06.857135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c9ade6-4448-4a4c-aa65-5f51b8d0791e admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:06.861634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ac620-c391-4644-817a-e19cb35da2a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:06.862289 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ac620-c391-4644-817a-e19cb35da2a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:06.862552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ac620-c391-4644-817a-e19cb35da2a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:06.862807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ac620-c391-4644-817a-e19cb35da2a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:06.874294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ac620-c391-4644-817a-e19cb35da2a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:06.875758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ac620-c391-4644-817a-e19cb35da2a8 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:06.885995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1272/2564] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:06.997165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078238b-b88a-4a91-a291-29235abdf40e None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:07.007678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078238b-b88a-4a91-a291-29235abdf40e None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:07.011245 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afcd19f-2ce6-4720-a186-7d504cfb581a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:07.011845 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcd19f-2ce6-4720-a186-7d504cfb581a service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:07.012119 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcd19f-2ce6-4720-a186-7d504cfb581a service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:07.012386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcd19f-2ce6-4720-a186-7d504cfb581a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:07.059181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:07.081971 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcd19f-2ce6-4720-a186-7d504cfb581a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:07.081971 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcd19f-2ce6-4720-a186-7d504cfb581a service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:07.090712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1293/2565] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:10.287795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ac620-c391-4644-817a-e19cb35da2a8 service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:10.352597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ac620-c391-4644-817a-e19cb35da2a8 service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:10.357097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:10.357722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:10.361656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:10.362459 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] PATH_INFO: `/v3/users/519b8e2496c8422ebe717b3d9940d9e8` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:10.363629 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=519b8e2496c8422ebe717b3d9940d9e8)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:10.365521 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:10.546905 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '519b8e2496c8422ebe717b3d9940d9e8'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:10.547467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '519b8e2496c8422ebe717b3d9940d9e8'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:10.619396 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '519b8e2496c8422ebe717b3d9940d9e8'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:10.627902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '519b8e2496c8422ebe717b3d9940d9e8'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:10.628478 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '519b8e2496c8422ebe717b3d9940d9e8'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:10.634373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:10.647927 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1273/2566] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:10 2022] DELETE /identity/v3/users/519b8e2496c8422ebe717b3d9940d9e8 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:10.652603 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ab553-2b50-4894-a2bb-e5b54cc8589c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:10.653105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ab553-2b50-4894-a2bb-e5b54cc8589c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:10.653300 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ab553-2b50-4894-a2bb-e5b54cc8589c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:10.674000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-0a1ab553-2b50-4894-a2bb-e5b54cc8589c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:10.675530 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0a1ab553-2b50-4894-a2bb-e5b54cc8589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:10.736394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1294/2567] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:10.746649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:10.748350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:10.810911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e96b6-60b0-4370-9a05-9c10975f4d0a admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:10.814717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc93b3-9d36-48ba-b7a4-6395ee1022ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:10.815337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc93b3-9d36-48ba-b7a4-6395ee1022ca service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:10.815615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc93b3-9d36-48ba-b7a4-6395ee1022ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:10.815883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc93b3-9d36-48ba-b7a4-6395ee1022ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:10.827529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc93b3-9d36-48ba-b7a4-6395ee1022ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:10.828938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc93b3-9d36-48ba-b7a4-6395ee1022ca service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:10.841058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1274/2568] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:11.285667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ab553-2b50-4894-a2bb-e5b54cc8589c None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:11.287376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0a1ab553-2b50-4894-a2bb-e5b54cc8589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:11.300067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ab553-2b50-4894-a2bb-e5b54cc8589c None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:11.305011 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:11.305643 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:11.305977 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:11.306249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] PATH_INFO: `/v3/projects/2c984014cb4044aa8fae1cca1553f7af` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:11.307360 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c984014cb4044aa8fae1cca1553f7af)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:11.309255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:11.415639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c984014cb4044aa8fae1cca1553f7af', 'request_id': 'req-db5e6a2b-6fb8-497d-bf05-fad54924fea3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:11.421121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c984014cb4044aa8fae1cca1553f7af', 'request_id': 'req-db5e6a2b-6fb8-497d-bf05-fad54924fea3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:11.427291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c984014cb4044aa8fae1cca1553f7af', 'request_id': 'req-db5e6a2b-6fb8-497d-bf05-fad54924fea3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:11.477854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] The token cache is being invalidate because project 2c984014cb4044aa8fae1cca1553f7af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:11.478205 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:11.478948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:11.480723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1295/2569] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:11 2022] DELETE /identity/v3/projects/2c984014cb4044aa8fae1cca1553f7af => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:11.484721 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc93b3-9d36-48ba-b7a4-6395ee1022ca service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:11.501185 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc93b3-9d36-48ba-b7a4-6395ee1022ca service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:11.564643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da7cad-7368-47ae-a5a5-635143efc989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:11.565547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da7cad-7368-47ae-a5a5-635143efc989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:11.565840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da7cad-7368-47ae-a5a5-635143efc989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:11.566127 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da7cad-7368-47ae-a5a5-635143efc989 admin admin] PATH_INFO: `/v3/users/b84aa7c3011b4238bb3ae00d712936ec` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:11.567265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da7cad-7368-47ae-a5a5-635143efc989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b84aa7c3011b4238bb3ae00d712936ec)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:11.569177 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da7cad-7368-47ae-a5a5-635143efc989 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:11.707645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-17da7cad-7368-47ae-a5a5-635143efc989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b84aa7c3011b4238bb3ae00d712936ec'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:11.708297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-17da7cad-7368-47ae-a5a5-635143efc989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b84aa7c3011b4238bb3ae00d712936ec'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:11.768414 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-17da7cad-7368-47ae-a5a5-635143efc989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b84aa7c3011b4238bb3ae00d712936ec'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:11.776006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-17da7cad-7368-47ae-a5a5-635143efc989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b84aa7c3011b4238bb3ae00d712936ec'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:11.776721 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-17da7cad-7368-47ae-a5a5-635143efc989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b84aa7c3011b4238bb3ae00d712936ec'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:11.797421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1275/2570] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:11 2022] DELETE /identity/v3/users/b84aa7c3011b4238bb3ae00d712936ec => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:12.197334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:12.199744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:12.213292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e6a2b-6fb8-497d-bf05-fad54924fea3 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:12.273718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf84d73-0e39-4e51-b2bd-b8b098e53023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:12.274436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf84d73-0e39-4e51-b2bd-b8b098e53023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:12.274724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf84d73-0e39-4e51-b2bd-b8b098e53023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:12.274996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf84d73-0e39-4e51-b2bd-b8b098e53023 admin admin] PATH_INFO: `/v3/projects/f817d9776f1740c88b766511ab6ba7fc` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:12.276115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf84d73-0e39-4e51-b2bd-b8b098e53023 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f817d9776f1740c88b766511ab6ba7fc)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:12.278210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf84d73-0e39-4e51-b2bd-b8b098e53023 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:12.285386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:12.398873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ecf84d73-0e39-4e51-b2bd-b8b098e53023 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f817d9776f1740c88b766511ab6ba7fc', 'request_id': 'req-ecf84d73-0e39-4e51-b2bd-b8b098e53023'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:12.405348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ecf84d73-0e39-4e51-b2bd-b8b098e53023 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f817d9776f1740c88b766511ab6ba7fc', 'request_id': 'req-ecf84d73-0e39-4e51-b2bd-b8b098e53023'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:12.412627 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ecf84d73-0e39-4e51-b2bd-b8b098e53023 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f817d9776f1740c88b766511ab6ba7fc', 'request_id': 'req-ecf84d73-0e39-4e51-b2bd-b8b098e53023'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:12.487593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ecf84d73-0e39-4e51-b2bd-b8b098e53023 admin admin] The token cache is being invalidate because project f817d9776f1740c88b766511ab6ba7fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:12.487880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ecf84d73-0e39-4e51-b2bd-b8b098e53023 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:12.488488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-ecf84d73-0e39-4e51-b2bd-b8b098e53023 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:12.490039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1296/2571] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:12 2022] DELETE /identity/v3/projects/f817d9776f1740c88b766511ab6ba7fc => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:12.494836 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39564af0-d37c-4610-b4d9-902d6122fb7b None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:12.496634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39564af0-d37c-4610-b4d9-902d6122fb7b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:12.496634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39564af0-d37c-4610-b4d9-902d6122fb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:12.515170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-39564af0-d37c-4610-b4d9-902d6122fb7b None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:12.516358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-39564af0-d37c-4610-b4d9-902d6122fb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:12.520576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:12.575309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1276/2572] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:12.584397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c97497-ad63-4254-9902-561d1c2ba0b4 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:12.584672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c97497-ad63-4254-9902-561d1c2ba0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:12.584941 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c97497-ad63-4254-9902-561d1c2ba0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:12.606164 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-f1c97497-ad63-4254-9902-561d1c2ba0b4 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:12.607288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f1c97497-ad63-4254-9902-561d1c2ba0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:12.615808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1297/2573] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:12.619985 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39564af0-d37c-4610-b4d9-902d6122fb7b None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:12.629672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39564af0-d37c-4610-b4d9-902d6122fb7b None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:12.634618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c2fda-b336-4c85-8fc4-c7cf0c67d5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:12.635311 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c2fda-b336-4c85-8fc4-c7cf0c67d5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:12.635592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c2fda-b336-4c85-8fc4-c7cf0c67d5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:12.635855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c2fda-b336-4c85-8fc4-c7cf0c67d5e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:12.636495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c2fda-b336-4c85-8fc4-c7cf0c67d5e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:50:12.636797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c2fda-b336-4c85-8fc4-c7cf0c67d5e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:12.638451 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c2fda-b336-4c85-8fc4-c7cf0c67d5e9 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:12.650832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1277/2574] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:50:12.655730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c97497-ad63-4254-9902-561d1c2ba0b4 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:12.665463 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c97497-ad63-4254-9902-561d1c2ba0b4 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:12.670150 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e36d41-8243-404a-ab1b-86a2337cfdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:12.671548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e36d41-8243-404a-ab1b-86a2337cfdd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:12.671813 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e36d41-8243-404a-ab1b-86a2337cfdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:12.672201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e36d41-8243-404a-ab1b-86a2337cfdd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:12.672874 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e36d41-8243-404a-ab1b-86a2337cfdd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:12.674776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e36d41-8243-404a-ab1b-86a2337cfdd4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:12.702236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c2fda-b336-4c85-8fc4-c7cf0c67d5e9 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:12.704227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c84c2fda-b336-4c85-8fc4-c7cf0c67d5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:12.762764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c2fda-b336-4c85-8fc4-c7cf0c67d5e9 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:12.767080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13746ebc-5d67-4320-9b8e-8ccf18bd9a0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:12.767682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13746ebc-5d67-4320-9b8e-8ccf18bd9a0b service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:12.768138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13746ebc-5d67-4320-9b8e-8ccf18bd9a0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:12.768402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13746ebc-5d67-4320-9b8e-8ccf18bd9a0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:12.771146 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-13746ebc-5d67-4320-9b8e-8ccf18bd9a0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:12.775248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1298/2575] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:12 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:12.779275 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e36d41-8243-404a-ab1b-86a2337cfdd4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:12.839600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13746ebc-5d67-4320-9b8e-8ccf18bd9a0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:12.841119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13746ebc-5d67-4320-9b8e-8ccf18bd9a0b service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:12.852666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1278/2576] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:50:12.857110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e36d41-8243-404a-ab1b-86a2337cfdd4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:12.861806 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e184fc-a2f5-40a1-8b1a-7ba66400e09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:12.862460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e184fc-a2f5-40a1-8b1a-7ba66400e09e admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:12.862731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e184fc-a2f5-40a1-8b1a-7ba66400e09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:12.862997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e184fc-a2f5-40a1-8b1a-7ba66400e09e admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:12.863655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e184fc-a2f5-40a1-8b1a-7ba66400e09e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:12.865562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e184fc-a2f5-40a1-8b1a-7ba66400e09e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:12.956574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1299/2577] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:12 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:12.958008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13746ebc-5d67-4320-9b8e-8ccf18bd9a0b service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:12.969639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13746ebc-5d67-4320-9b8e-8ccf18bd9a0b service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:12.974421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96cfe8-4458-49a0-b6fb-5b49b4cd32b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:12.975183 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96cfe8-4458-49a0-b6fb-5b49b4cd32b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:12.975896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96cfe8-4458-49a0-b6fb-5b49b4cd32b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:12.975896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96cfe8-4458-49a0-b6fb-5b49b4cd32b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:12.976362 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96cfe8-4458-49a0-b6fb-5b49b4cd32b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:12.978155 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96cfe8-4458-49a0-b6fb-5b49b4cd32b0 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:12.989395 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1279/2578] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:12.993295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e184fc-a2f5-40a1-8b1a-7ba66400e09e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:13.004427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e184fc-a2f5-40a1-8b1a-7ba66400e09e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:13.011216 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceefb91-eaf6-459c-a027-c6a8d3ccf800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:13.012162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceefb91-eaf6-459c-a027-c6a8d3ccf800 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:13.012505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceefb91-eaf6-459c-a027-c6a8d3ccf800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:13.012780 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceefb91-eaf6-459c-a027-c6a8d3ccf800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:13.013432 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceefb91-eaf6-459c-a027-c6a8d3ccf800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:13.015824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceefb91-eaf6-459c-a027-c6a8d3ccf800 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:13.043522 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1300/2579] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:13.048610 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96cfe8-4458-49a0-b6fb-5b49b4cd32b0 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:13.064731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96cfe8-4458-49a0-b6fb-5b49b4cd32b0 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:13.071200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13788-8e59-447b-9d25-855c9844868d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:13.072359 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13788-8e59-447b-9d25-855c9844868d admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:13.072636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13788-8e59-447b-9d25-855c9844868d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:13.072907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13788-8e59-447b-9d25-855c9844868d admin admin] PATH_INFO: `/v3/projects/95d5e9eeb37b46e4ba94bcce1471b694/users/4f7e5cf724564900ac5f55a6f5ceb0b4/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:13.098216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa13788-8e59-447b-9d25-855c9844868d admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d5e9eeb37b46e4ba94bcce1471b694, user_id=4f7e5cf724564900ac5f55a6f5ceb0b4, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:13.101397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa13788-8e59-447b-9d25-855c9844868d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:13.196946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1280/2580] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:13 2022] PUT /identity/v3/projects/95d5e9eeb37b46e4ba94bcce1471b694/users/4f7e5cf724564900ac5f55a6f5ceb0b4/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:13.207580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d074f4-ece3-4109-814f-d153bf642597 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:13.208056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d074f4-ece3-4109-814f-d153bf642597 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:13.208161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d074f4-ece3-4109-814f-d153bf642597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:13.244183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-e5d074f4-ece3-4109-814f-d153bf642597 None None] MFA Rules not processed for user `4f7e5cf724564900ac5f55a6f5ceb0b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:13.245410 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e5d074f4-ece3-4109-814f-d153bf642597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:13.296206 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:13.385684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1301/2581] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:13.395359 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13788-8e59-447b-9d25-855c9844868d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:13.397444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6aa13788-8e59-447b-9d25-855c9844868d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:13.422300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:13.466064 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13788-8e59-447b-9d25-855c9844868d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:13.473223 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e198be-4d65-4517-b08a-51a6bde0dbe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:13.473886 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e198be-4d65-4517-b08a-51a6bde0dbe2 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:13.474172 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e198be-4d65-4517-b08a-51a6bde0dbe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:13.490316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e198be-4d65-4517-b08a-51a6bde0dbe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:13.506270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e198be-4d65-4517-b08a-51a6bde0dbe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:13.511510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e198be-4d65-4517-b08a-51a6bde0dbe2 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:13.537495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1281/2582] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:13.547887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d074f4-ece3-4109-814f-d153bf642597 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:13.654152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d074f4-ece3-4109-814f-d153bf642597 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:13.660123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:13.660977 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:13.661658 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:13.662118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] PATH_INFO: `/v3/users/4f7e5cf724564900ac5f55a6f5ceb0b4` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:13.663506 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f7e5cf724564900ac5f55a6f5ceb0b4)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:13.665814 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:13.957221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f7e5cf724564900ac5f55a6f5ceb0b4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:13.957912 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f7e5cf724564900ac5f55a6f5ceb0b4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:14.031730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f7e5cf724564900ac5f55a6f5ceb0b4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:14.032479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f7e5cf724564900ac5f55a6f5ceb0b4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:14.040907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f7e5cf724564900ac5f55a6f5ceb0b4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:14.073144 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1302/2583] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:13 2022] DELETE /identity/v3/users/4f7e5cf724564900ac5f55a6f5ceb0b4 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:50:14.083098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533dbc3-2466-4272-85e6-f94a8c7cda5b None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:14.083397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533dbc3-2466-4272-85e6-f94a8c7cda5b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:14.083662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533dbc3-2466-4272-85e6-f94a8c7cda5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:14.120001 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-5533dbc3-2466-4272-85e6-f94a8c7cda5b None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:14.121249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5533dbc3-2466-4272-85e6-f94a8c7cda5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:14.224885 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1282/2584] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:14.234998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:14.238007 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:14.313036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc6a2d-11fd-487d-9195-c725844cdc39 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:14.316794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e1c36-2e4e-4a3e-833b-46ee688a3966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:14.317524 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e1c36-2e4e-4a3e-833b-46ee688a3966 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:14.317900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e1c36-2e4e-4a3e-833b-46ee688a3966 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:14.318244 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e1c36-2e4e-4a3e-833b-46ee688a3966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:14.328668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e1c36-2e4e-4a3e-833b-46ee688a3966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:14.330463 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579e1c36-2e4e-4a3e-833b-46ee688a3966 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:14.341140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1303/2585] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:14.724221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:14.976890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533dbc3-2466-4272-85e6-f94a8c7cda5b None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:14.978950 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5533dbc3-2466-4272-85e6-f94a8c7cda5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:15.001763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533dbc3-2466-4272-85e6-f94a8c7cda5b None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:15.006707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c21664-60cd-459f-920c-9970ddcbf8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:15.007463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c21664-60cd-459f-920c-9970ddcbf8b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:15.007733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c21664-60cd-459f-920c-9970ddcbf8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:15.007997 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c21664-60cd-459f-920c-9970ddcbf8b3 admin admin] PATH_INFO: `/v3/projects/95d5e9eeb37b46e4ba94bcce1471b694` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:15.009114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c21664-60cd-459f-920c-9970ddcbf8b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d5e9eeb37b46e4ba94bcce1471b694)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:15.011211 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c21664-60cd-459f-920c-9970ddcbf8b3 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:15.151578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c8c21664-60cd-459f-920c-9970ddcbf8b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d5e9eeb37b46e4ba94bcce1471b694', 'request_id': 'req-c8c21664-60cd-459f-920c-9970ddcbf8b3'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:15.158455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c8c21664-60cd-459f-920c-9970ddcbf8b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d5e9eeb37b46e4ba94bcce1471b694', 'request_id': 'req-c8c21664-60cd-459f-920c-9970ddcbf8b3'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:15.165130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c8c21664-60cd-459f-920c-9970ddcbf8b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d5e9eeb37b46e4ba94bcce1471b694', 'request_id': 'req-c8c21664-60cd-459f-920c-9970ddcbf8b3'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:15.249428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c8c21664-60cd-459f-920c-9970ddcbf8b3 admin admin] The token cache is being invalidate because project 95d5e9eeb37b46e4ba94bcce1471b694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:15.249788 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c8c21664-60cd-459f-920c-9970ddcbf8b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:15.255442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c8c21664-60cd-459f-920c-9970ddcbf8b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:15.255442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1283/2586] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:14 2022] DELETE /identity/v3/projects/95d5e9eeb37b46e4ba94bcce1471b694 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:15.257027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a307ad6-02f8-4efb-8175-b21411cf757e None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:15.257476 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a307ad6-02f8-4efb-8175-b21411cf757e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:15.257588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a307ad6-02f8-4efb-8175-b21411cf757e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:15.280093 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-1a307ad6-02f8-4efb-8175-b21411cf757e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:15.281324 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-1a307ad6-02f8-4efb-8175-b21411cf757e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:15.345103 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1304/2587] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:50:15.352908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0de17-31fb-427e-ba67-4747d749fc27 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:15.353188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0de17-31fb-427e-ba67-4747d749fc27 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:15.353457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0de17-31fb-427e-ba67-4747d749fc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:15.374972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-a3f0de17-31fb-427e-ba67-4747d749fc27 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:15.376213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a3f0de17-31fb-427e-ba67-4747d749fc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:15.384290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1284/2588] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:15.388544 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a307ad6-02f8-4efb-8175-b21411cf757e None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:15.399529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a307ad6-02f8-4efb-8175-b21411cf757e None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:15.406700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd47bb8-d49a-4e3c-aa75-338c580296f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:15.407568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd47bb8-d49a-4e3c-aa75-338c580296f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:15.407844 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd47bb8-d49a-4e3c-aa75-338c580296f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:15.408106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd47bb8-d49a-4e3c-aa75-338c580296f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:15.408908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd47bb8-d49a-4e3c-aa75-338c580296f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:50:15.409212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd47bb8-d49a-4e3c-aa75-338c580296f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:15.410848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd47bb8-d49a-4e3c-aa75-338c580296f6 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:15.427243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1305/2589] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:50:15.428622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0de17-31fb-427e-ba67-4747d749fc27 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:15.438398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0de17-31fb-427e-ba67-4747d749fc27 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:15.443270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7f2c5-51bb-409d-9e82-a62ef03d5d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:15.443873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7f2c5-51bb-409d-9e82-a62ef03d5d7c admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:15.444154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7f2c5-51bb-409d-9e82-a62ef03d5d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:15.444420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7f2c5-51bb-409d-9e82-a62ef03d5d7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:15.445052 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7f2c5-51bb-409d-9e82-a62ef03d5d7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:15.447035 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c7f2c5-51bb-409d-9e82-a62ef03d5d7c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:15.540593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1285/2590] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:15 2022] POST /identity/v3/projects => generated 361 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:15.546315 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd47bb8-d49a-4e3c-aa75-338c580296f6 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:15.557151 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd47bb8-d49a-4e3c-aa75-338c580296f6 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:15.614710 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5a426-1f21-465c-bdf2-9bd7a55a688f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:15.615312 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5a426-1f21-465c-bdf2-9bd7a55a688f admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:15.615589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5a426-1f21-465c-bdf2-9bd7a55a688f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:15.615857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5a426-1f21-465c-bdf2-9bd7a55a688f admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:15.616487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b5a426-1f21-465c-bdf2-9bd7a55a688f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:15.618412 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b5a426-1f21-465c-bdf2-9bd7a55a688f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:15.687152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1306/2591] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:15 2022] POST /identity/v3/users => generated 281 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:50:15.688313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7f2c5-51bb-409d-9e82-a62ef03d5d7c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:15.698635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7f2c5-51bb-409d-9e82-a62ef03d5d7c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:15.703475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699a329-44f5-4fc2-b2ed-c19fe23cadae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:15.704092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699a329-44f5-4fc2-b2ed-c19fe23cadae admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:15.704426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699a329-44f5-4fc2-b2ed-c19fe23cadae admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:15.704641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699a329-44f5-4fc2-b2ed-c19fe23cadae admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:15.705285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699a329-44f5-4fc2-b2ed-c19fe23cadae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:15.706901 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699a329-44f5-4fc2-b2ed-c19fe23cadae admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:15.720877 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1286/2592] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:15.722127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5a426-1f21-465c-bdf2-9bd7a55a688f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:15.732476 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5a426-1f21-465c-bdf2-9bd7a55a688f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:15.738129 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2479e724-eea7-4e9c-bc26-68d395a1b96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:15.738807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479e724-eea7-4e9c-bc26-68d395a1b96f admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:15.739022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479e724-eea7-4e9c-bc26-68d395a1b96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:15.739301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2479e724-eea7-4e9c-bc26-68d395a1b96f admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:15.739942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479e724-eea7-4e9c-bc26-68d395a1b96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:15.741544 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2479e724-eea7-4e9c-bc26-68d395a1b96f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:15.756907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1307/2593] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:15.758149 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699a329-44f5-4fc2-b2ed-c19fe23cadae admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:15.763474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:15.769236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699a329-44f5-4fc2-b2ed-c19fe23cadae admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:15.773989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffa6d8-20c9-4a24-aae1-5fc533c01e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:15.774637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa6d8-20c9-4a24-aae1-5fc533c01e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:15.774911 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa6d8-20c9-4a24-aae1-5fc533c01e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:15.775180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffa6d8-20c9-4a24-aae1-5fc533c01e86 admin admin] PATH_INFO: `/v3/projects/285d6cb712ef47dba147ec277590482d/users/e74a0dcac3a9459b9b9d7e88bc1c0167/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:15.790435 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffa6d8-20c9-4a24-aae1-5fc533c01e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=285d6cb712ef47dba147ec277590482d, user_id=e74a0dcac3a9459b9b9d7e88bc1c0167, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:15.793165 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffa6d8-20c9-4a24-aae1-5fc533c01e86 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:15.878042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1287/2594] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:15 2022] PUT /identity/v3/projects/285d6cb712ef47dba147ec277590482d/users/e74a0dcac3a9459b9b9d7e88bc1c0167/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:15.889553 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61995c38-b582-4c38-a414-416913796209 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:15.889948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61995c38-b582-4c38-a414-416913796209 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:15.890154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61995c38-b582-4c38-a414-416913796209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:15.932813 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-61995c38-b582-4c38-a414-416913796209 None None] MFA Rules not processed for user `e74a0dcac3a9459b9b9d7e88bc1c0167`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:15.934197 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-61995c38-b582-4c38-a414-416913796209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:16.030483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:16.048643 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1308/2595] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:16.098300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffa6d8-20c9-4a24-aae1-5fc533c01e86 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:16.100008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3bffa6d8-20c9-4a24-aae1-5fc533c01e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:16.157643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffa6d8-20c9-4a24-aae1-5fc533c01e86 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:16.161303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774173c4-19c4-4043-b66d-7639b6e9bf48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:16.162190 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774173c4-19c4-4043-b66d-7639b6e9bf48 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:16.162463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774173c4-19c4-4043-b66d-7639b6e9bf48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:16.162730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774173c4-19c4-4043-b66d-7639b6e9bf48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:16.171472 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774173c4-19c4-4043-b66d-7639b6e9bf48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:16.173223 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774173c4-19c4-4043-b66d-7639b6e9bf48 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:16.182691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1288/2596] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:16.198964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61995c38-b582-4c38-a414-416913796209 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:16.265859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61995c38-b582-4c38-a414-416913796209 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:16.270560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:16.271208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:16.271481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:16.271751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] PATH_INFO: `/v3/users/e74a0dcac3a9459b9b9d7e88bc1c0167` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:16.272857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e74a0dcac3a9459b9b9d7e88bc1c0167)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:16.275228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:16.479267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e74a0dcac3a9459b9b9d7e88bc1c0167'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:16.479907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e74a0dcac3a9459b9b9d7e88bc1c0167'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:16.522538 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e74a0dcac3a9459b9b9d7e88bc1c0167'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:16.523242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e74a0dcac3a9459b9b9d7e88bc1c0167'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:16.531363 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e74a0dcac3a9459b9b9d7e88bc1c0167'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:16.551833 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1309/2597] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:16 2022] DELETE /identity/v3/users/e74a0dcac3a9459b9b9d7e88bc1c0167 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:50:16.557041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8c51d-d5a2-41dd-a45d-b30d6594e641 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:16.557330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8c51d-d5a2-41dd-a45d-b30d6594e641 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:16.557600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8c51d-d5a2-41dd-a45d-b30d6594e641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:16.582415 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-3ba8c51d-d5a2-41dd-a45d-b30d6594e641 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:16.583794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3ba8c51d-d5a2-41dd-a45d-b30d6594e641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:16.644932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1289/2598] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:16.654347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:16.655788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:16.721217 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99305ce2-dee9-4550-8eb4-e9dd5a6f6cc9 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:16.724954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c2974-7687-46fd-84d3-6446f50e476d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:16.725771 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2974-7687-46fd-84d3-6446f50e476d service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:16.726140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2974-7687-46fd-84d3-6446f50e476d service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:16.726481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c2974-7687-46fd-84d3-6446f50e476d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:16.736281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c2974-7687-46fd-84d3-6446f50e476d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:16.737705 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c2974-7687-46fd-84d3-6446f50e476d service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:16.747152 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1310/2599] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:17.606061 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8c51d-d5a2-41dd-a45d-b30d6594e641 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:17.608850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3ba8c51d-d5a2-41dd-a45d-b30d6594e641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:17.627373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8c51d-d5a2-41dd-a45d-b30d6594e641 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:17.636032 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4088d6-3f65-4be7-bd8d-a1d2db754470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:17.636032 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4088d6-3f65-4be7-bd8d-a1d2db754470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:17.636032 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4088d6-3f65-4be7-bd8d-a1d2db754470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:17.636032 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4088d6-3f65-4be7-bd8d-a1d2db754470 admin admin] PATH_INFO: `/v3/projects/285d6cb712ef47dba147ec277590482d` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:17.638918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4088d6-3f65-4be7-bd8d-a1d2db754470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=285d6cb712ef47dba147ec277590482d)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:17.641339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4088d6-3f65-4be7-bd8d-a1d2db754470 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:17.776589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3a4088d6-3f65-4be7-bd8d-a1d2db754470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285d6cb712ef47dba147ec277590482d', 'request_id': 'req-3a4088d6-3f65-4be7-bd8d-a1d2db754470'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:17.785505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3a4088d6-3f65-4be7-bd8d-a1d2db754470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285d6cb712ef47dba147ec277590482d', 'request_id': 'req-3a4088d6-3f65-4be7-bd8d-a1d2db754470'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:17.792934 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3a4088d6-3f65-4be7-bd8d-a1d2db754470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285d6cb712ef47dba147ec277590482d', 'request_id': 'req-3a4088d6-3f65-4be7-bd8d-a1d2db754470'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:17.886638 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3a4088d6-3f65-4be7-bd8d-a1d2db754470 admin admin] The token cache is being invalidate because project 285d6cb712ef47dba147ec277590482d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:17.887239 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3a4088d6-3f65-4be7-bd8d-a1d2db754470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:17.888067 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3a4088d6-3f65-4be7-bd8d-a1d2db754470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:17.890746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1290/2600] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:17 2022] DELETE /identity/v3/projects/285d6cb712ef47dba147ec277590482d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:17.897794 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25994563-477b-4254-947f-f542449cc90a None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:17.898388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25994563-477b-4254-947f-f542449cc90a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:17.898388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25994563-477b-4254-947f-f542449cc90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:17.929424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-25994563-477b-4254-947f-f542449cc90a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:17.930835 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-25994563-477b-4254-947f-f542449cc90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:17.941188 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:18.002613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1311/2601] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:18.010708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d769ac5-83df-40e8-a714-a42c1b4491ff None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:18.010996 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d769ac5-83df-40e8-a714-a42c1b4491ff None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:18.011630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d769ac5-83df-40e8-a714-a42c1b4491ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:18.022893 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:18.037141 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-7d769ac5-83df-40e8-a714-a42c1b4491ff None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:18.038528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7d769ac5-83df-40e8-a714-a42c1b4491ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:18.051471 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1291/2602] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:18.052576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25994563-477b-4254-947f-f542449cc90a None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:18.064252 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25994563-477b-4254-947f-f542449cc90a None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:18.069947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca86be9-49e8-4eeb-bd99-bcd7d7df346f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:18.070670 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca86be9-49e8-4eeb-bd99-bcd7d7df346f admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:18.071214 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca86be9-49e8-4eeb-bd99-bcd7d7df346f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:18.071540 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca86be9-49e8-4eeb-bd99-bcd7d7df346f admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:18.072343 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca86be9-49e8-4eeb-bd99-bcd7d7df346f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:50:18.073687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca86be9-49e8-4eeb-bd99-bcd7d7df346f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:18.074787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca86be9-49e8-4eeb-bd99-bcd7d7df346f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:18.093797 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1312/2603] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:50:18.095462 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d769ac5-83df-40e8-a714-a42c1b4491ff None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:18.105420 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d769ac5-83df-40e8-a714-a42c1b4491ff None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:18.111160 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119e50e-e4f2-472d-b527-1df90f1eb2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:18.112242 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119e50e-e4f2-472d-b527-1df90f1eb2e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:18.112525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119e50e-e4f2-472d-b527-1df90f1eb2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:18.113646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119e50e-e4f2-472d-b527-1df90f1eb2e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:18.113646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119e50e-e4f2-472d-b527-1df90f1eb2e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:18.115922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119e50e-e4f2-472d-b527-1df90f1eb2e7 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:18.218747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1292/2604] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:18 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:18.222854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca86be9-49e8-4eeb-bd99-bcd7d7df346f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:18.236159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca86be9-49e8-4eeb-bd99-bcd7d7df346f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:18.313917 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f154b97-76f8-4d05-8228-89bffce39e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:18.314859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f154b97-76f8-4d05-8228-89bffce39e22 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:18.315333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f154b97-76f8-4d05-8228-89bffce39e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:18.315865 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f154b97-76f8-4d05-8228-89bffce39e22 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:18.316868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f154b97-76f8-4d05-8228-89bffce39e22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:18.320455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f154b97-76f8-4d05-8228-89bffce39e22 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:18.414918 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1313/2605] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:18 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:18.419154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119e50e-e4f2-472d-b527-1df90f1eb2e7 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:18.430786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119e50e-e4f2-472d-b527-1df90f1eb2e7 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:18.437008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416f043-c771-4e50-8160-cac7fa9b87fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:18.437008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416f043-c771-4e50-8160-cac7fa9b87fe admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:18.437008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416f043-c771-4e50-8160-cac7fa9b87fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:18.437008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1416f043-c771-4e50-8160-cac7fa9b87fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:18.437564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416f043-c771-4e50-8160-cac7fa9b87fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:18.439262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1416f043-c771-4e50-8160-cac7fa9b87fe admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:18.451247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1293/2606] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:18.455492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f154b97-76f8-4d05-8228-89bffce39e22 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:18.466165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f154b97-76f8-4d05-8228-89bffce39e22 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:18.473697 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fc661-366e-40b4-8926-108d5f08f602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:18.474356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fc661-366e-40b4-8926-108d5f08f602 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:18.474631 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fc661-366e-40b4-8926-108d5f08f602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:18.475989 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fc661-366e-40b4-8926-108d5f08f602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:18.477858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fc661-366e-40b4-8926-108d5f08f602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:18.479319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fc661-366e-40b4-8926-108d5f08f602 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:18.494585 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1314/2607] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:18.499479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416f043-c771-4e50-8160-cac7fa9b87fe admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:18.509509 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1416f043-c771-4e50-8160-cac7fa9b87fe admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:18.514308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49db068-b5d8-4d8a-ab74-ee7db05b3faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:18.514955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49db068-b5d8-4d8a-ab74-ee7db05b3faa admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:18.515231 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49db068-b5d8-4d8a-ab74-ee7db05b3faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:18.515505 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49db068-b5d8-4d8a-ab74-ee7db05b3faa admin admin] PATH_INFO: `/v3/projects/cfb29f2890524c61a0db4567fcd40200/users/6f702d21354748158c7b79e7e3165b7f/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:18.532519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49db068-b5d8-4d8a-ab74-ee7db05b3faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb29f2890524c61a0db4567fcd40200, user_id=6f702d21354748158c7b79e7e3165b7f, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:18.535299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49db068-b5d8-4d8a-ab74-ee7db05b3faa admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:18.654220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1294/2608] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:18 2022] PUT /identity/v3/projects/cfb29f2890524c61a0db4567fcd40200/users/6f702d21354748158c7b79e7e3165b7f/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:18.665441 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd59132-2a66-474c-bc6f-2b2271093fbf None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:18.665766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd59132-2a66-474c-bc6f-2b2271093fbf None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:18.666024 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd59132-2a66-474c-bc6f-2b2271093fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:18.697138 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-7cd59132-2a66-474c-bc6f-2b2271093fbf None None] MFA Rules not processed for user `6f702d21354748158c7b79e7e3165b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:18.698568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7cd59132-2a66-474c-bc6f-2b2271093fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:18.787710 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1315/2609] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:18.800206 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49db068-b5d8-4d8a-ab74-ee7db05b3faa admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:18.805621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-b49db068-b5d8-4d8a-ab74-ee7db05b3faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:18.868628 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49db068-b5d8-4d8a-ab74-ee7db05b3faa admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:18.873024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c442775-a447-40ca-b3cd-5717047a6ec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:18.873441 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c442775-a447-40ca-b3cd-5717047a6ec7 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:18.874685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c442775-a447-40ca-b3cd-5717047a6ec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:18.875114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c442775-a447-40ca-b3cd-5717047a6ec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:18.884176 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c442775-a447-40ca-b3cd-5717047a6ec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:18.885924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c442775-a447-40ca-b3cd-5717047a6ec7 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:18.895803 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1295/2610] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:20.206937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd59132-2a66-474c-bc6f-2b2271093fbf None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:20.208560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7cd59132-2a66-474c-bc6f-2b2271093fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:20.219387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:20.269046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd59132-2a66-474c-bc6f-2b2271093fbf None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:20.272758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469a0b3-1eb8-451a-b34a-9ce92e10202c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:20.273370 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469a0b3-1eb8-451a-b34a-9ce92e10202c service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:20.273647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469a0b3-1eb8-451a-b34a-9ce92e10202c service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:20.275120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469a0b3-1eb8-451a-b34a-9ce92e10202c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:20.275941 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d469a0b3-1eb8-451a-b34a-9ce92e10202c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:20.340447 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469a0b3-1eb8-451a-b34a-9ce92e10202c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:20.342045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469a0b3-1eb8-451a-b34a-9ce92e10202c service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:20.353598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1316/2611] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:50:22.258133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c442775-a447-40ca-b3cd-5717047a6ec7 service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:22.275151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c442775-a447-40ca-b3cd-5717047a6ec7 service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:22.280794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:22.284768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:22.284768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:22.284768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] PATH_INFO: `/v3/users/6f702d21354748158c7b79e7e3165b7f` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:22.284768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f702d21354748158c7b79e7e3165b7f)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:22.285810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:22.325343 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469a0b3-1eb8-451a-b34a-9ce92e10202c service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:22.326213 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d469a0b3-1eb8-451a-b34a-9ce92e10202c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:22.339133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469a0b3-1eb8-451a-b34a-9ce92e10202c service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:22.344316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c495a-d006-4891-b6ef-3d7b7db09f0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:22.349484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c495a-d006-4891-b6ef-3d7b7db09f0e service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:22.349484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c495a-d006-4891-b6ef-3d7b7db09f0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:22.349484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c495a-d006-4891-b6ef-3d7b7db09f0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:22.349484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0a6c495a-d006-4891-b6ef-3d7b7db09f0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:22.362624 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c495a-d006-4891-b6ef-3d7b7db09f0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:22.364079 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c495a-d006-4891-b6ef-3d7b7db09f0e service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:22.377132 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1317/2612] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:50:22.559510 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f702d21354748158c7b79e7e3165b7f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:22.560316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f702d21354748158c7b79e7e3165b7f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:22.643175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f702d21354748158c7b79e7e3165b7f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:22.652036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f702d21354748158c7b79e7e3165b7f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:22.653243 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f702d21354748158c7b79e7e3165b7f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:22.678774 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1296/2613] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:22 2022] DELETE /identity/v3/users/6f702d21354748158c7b79e7e3165b7f => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:22.683279 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51191ee8-3038-4c06-a0b2-23481c8b278a None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:22.683515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51191ee8-3038-4c06-a0b2-23481c8b278a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:22.683819 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51191ee8-3038-4c06-a0b2-23481c8b278a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:22.704429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-51191ee8-3038-4c06-a0b2-23481c8b278a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:22.705716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-51191ee8-3038-4c06-a0b2-23481c8b278a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:22.785745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1318/2614] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:22.795979 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:22.797646 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:22.834373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:22.889661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71120c3-bf06-4e3b-a9b6-9deb8592f00d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:22.895609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e3eab-7958-4d24-8fb2-835da613346a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:22.896467 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e3eab-7958-4d24-8fb2-835da613346a service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:22.896915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e3eab-7958-4d24-8fb2-835da613346a service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:22.897280 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e3eab-7958-4d24-8fb2-835da613346a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:22.907643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e3eab-7958-4d24-8fb2-835da613346a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:22.909201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e3eab-7958-4d24-8fb2-835da613346a service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:22.930315 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1297/2615] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:23.885984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51191ee8-3038-4c06-a0b2-23481c8b278a None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:23.887561 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-51191ee8-3038-4c06-a0b2-23481c8b278a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:23.901268 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51191ee8-3038-4c06-a0b2-23481c8b278a None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:23.907958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606807d-8f73-4e6a-8f53-299ced92dcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:23.908597 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606807d-8f73-4e6a-8f53-299ced92dcb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:23.908868 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606807d-8f73-4e6a-8f53-299ced92dcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:23.909133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606807d-8f73-4e6a-8f53-299ced92dcb1 admin admin] PATH_INFO: `/v3/projects/cfb29f2890524c61a0db4567fcd40200` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:23.910704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606807d-8f73-4e6a-8f53-299ced92dcb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfb29f2890524c61a0db4567fcd40200)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:23.914620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606807d-8f73-4e6a-8f53-299ced92dcb1 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:24.033165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:24.040205 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9606807d-8f73-4e6a-8f53-299ced92dcb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfb29f2890524c61a0db4567fcd40200', 'request_id': 'req-9606807d-8f73-4e6a-8f53-299ced92dcb1'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:24.046025 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9606807d-8f73-4e6a-8f53-299ced92dcb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfb29f2890524c61a0db4567fcd40200', 'request_id': 'req-9606807d-8f73-4e6a-8f53-299ced92dcb1'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:24.052700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9606807d-8f73-4e6a-8f53-299ced92dcb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfb29f2890524c61a0db4567fcd40200', 'request_id': 'req-9606807d-8f73-4e6a-8f53-299ced92dcb1'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:24.129163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9606807d-8f73-4e6a-8f53-299ced92dcb1 admin admin] The token cache is being invalidate because project cfb29f2890524c61a0db4567fcd40200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:24.129642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9606807d-8f73-4e6a-8f53-299ced92dcb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:24.130336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9606807d-8f73-4e6a-8f53-299ced92dcb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:24.132910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1319/2616] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:23 2022] DELETE /identity/v3/projects/cfb29f2890524c61a0db4567fcd40200 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:24.136981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7a637-6f0f-45e1-bc3b-69c7ea869cd4 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:24.137519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7a637-6f0f-45e1-bc3b-69c7ea869cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:24.137519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7a637-6f0f-45e1-bc3b-69c7ea869cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:24.158984 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-23e7a637-6f0f-45e1-bc3b-69c7ea869cd4 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:24.160318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-23e7a637-6f0f-45e1-bc3b-69c7ea869cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:24.225566 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1298/2617] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:24.235192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1230ba-0cf6-483d-ac88-1abb8b908bd7 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:24.235471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1230ba-0cf6-483d-ac88-1abb8b908bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:24.235873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1230ba-0cf6-483d-ac88-1abb8b908bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:24.255815 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-dd1230ba-0cf6-483d-ac88-1abb8b908bd7 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:24.257060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-dd1230ba-0cf6-483d-ac88-1abb8b908bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:24.266451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1320/2618] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:50:24.270809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7a637-6f0f-45e1-bc3b-69c7ea869cd4 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:24.282691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7a637-6f0f-45e1-bc3b-69c7ea869cd4 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:24.288177 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d550de-e2bf-4315-825c-7522f49018c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:24.289005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d550de-e2bf-4315-825c-7522f49018c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:24.289318 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d550de-e2bf-4315-825c-7522f49018c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:24.289593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d550de-e2bf-4315-825c-7522f49018c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:24.290313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d550de-e2bf-4315-825c-7522f49018c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:50:24.290616 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d550de-e2bf-4315-825c-7522f49018c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:24.292201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d550de-e2bf-4315-825c-7522f49018c8 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:24.304799 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1299/2619] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:50:24.309355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1230ba-0cf6-483d-ac88-1abb8b908bd7 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:24.320365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1230ba-0cf6-483d-ac88-1abb8b908bd7 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:24.325290 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bd8c5-986d-41c9-83ab-9a1a58e20baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:24.325998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bd8c5-986d-41c9-83ab-9a1a58e20baf admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:24.326271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bd8c5-986d-41c9-83ab-9a1a58e20baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:24.327045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bd8c5-986d-41c9-83ab-9a1a58e20baf admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:24.327747 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bd8c5-986d-41c9-83ab-9a1a58e20baf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:24.329994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bd8c5-986d-41c9-83ab-9a1a58e20baf admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:24.425928 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1321/2620] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:24 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:24.430238 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d550de-e2bf-4315-825c-7522f49018c8 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:24.441862 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d550de-e2bf-4315-825c-7522f49018c8 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:24.501935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbeaba-8c59-4532-b86f-60b40f7da46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:24.502548 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbeaba-8c59-4532-b86f-60b40f7da46c admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:24.502822 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbeaba-8c59-4532-b86f-60b40f7da46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:24.503090 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbbeaba-8c59-4532-b86f-60b40f7da46c admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:24.503718 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbeaba-8c59-4532-b86f-60b40f7da46c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:24.505650 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbbeaba-8c59-4532-b86f-60b40f7da46c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:24.604651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1300/2621] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:24 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:50:24.609576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bd8c5-986d-41c9-83ab-9a1a58e20baf admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:24.620825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bd8c5-986d-41c9-83ab-9a1a58e20baf admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:24.626830 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333d974-7f46-4cda-bb5c-cd95ad9fc7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:24.627442 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333d974-7f46-4cda-bb5c-cd95ad9fc7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:24.627716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333d974-7f46-4cda-bb5c-cd95ad9fc7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:24.627984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333d974-7f46-4cda-bb5c-cd95ad9fc7d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:24.628621 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333d974-7f46-4cda-bb5c-cd95ad9fc7d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:24.630238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333d974-7f46-4cda-bb5c-cd95ad9fc7d6 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:24.644618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1322/2622] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:24.646111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbeaba-8c59-4532-b86f-60b40f7da46c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:24.657965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbbeaba-8c59-4532-b86f-60b40f7da46c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:24.662924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c666f4-4744-43d2-ad7c-9c8a59e25e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:24.663527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c666f4-4744-43d2-ad7c-9c8a59e25e31 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:24.663805 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c666f4-4744-43d2-ad7c-9c8a59e25e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:24.664072 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c666f4-4744-43d2-ad7c-9c8a59e25e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:24.664712 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c666f4-4744-43d2-ad7c-9c8a59e25e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:24.666331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c666f4-4744-43d2-ad7c-9c8a59e25e31 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:24.680573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1301/2623] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:24.684985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333d974-7f46-4cda-bb5c-cd95ad9fc7d6 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:24.696869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333d974-7f46-4cda-bb5c-cd95ad9fc7d6 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:24.704736 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f4b74-265a-4641-be67-6ae7e211991c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:24.705401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f4b74-265a-4641-be67-6ae7e211991c admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:24.705679 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f4b74-265a-4641-be67-6ae7e211991c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:24.706273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f4b74-265a-4641-be67-6ae7e211991c admin admin] PATH_INFO: `/v3/projects/d0f3aae0f9814d189d002373cc873f42/users/7092b595cffa4ba48d45982165ac27b0/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:24.722729 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f4b74-265a-4641-be67-6ae7e211991c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f3aae0f9814d189d002373cc873f42, user_id=7092b595cffa4ba48d45982165ac27b0, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:24.725568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f4b74-265a-4641-be67-6ae7e211991c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:24.810634 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1323/2624] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:24 2022] PUT /identity/v3/projects/d0f3aae0f9814d189d002373cc873f42/users/7092b595cffa4ba48d45982165ac27b0/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:50:24.816558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fd838-4c49-4ee3-838d-c2a683a499ae None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:24.816558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fd838-4c49-4ee3-838d-c2a683a499ae None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:24.817195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fd838-4c49-4ee3-838d-c2a683a499ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:24.844671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-946fd838-4c49-4ee3-838d-c2a683a499ae None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:24.846383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-946fd838-4c49-4ee3-838d-c2a683a499ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:24.875960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:24.944221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1302/2625] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:24.967381 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f4b74-265a-4641-be67-6ae7e211991c admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:24.969479 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2f5f4b74-265a-4641-be67-6ae7e211991c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:25.050317 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f4b74-265a-4641-be67-6ae7e211991c admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:25.055438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac64737-efcd-4690-a7c6-77cca53d6f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:25.056217 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac64737-efcd-4690-a7c6-77cca53d6f22 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:25.056500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac64737-efcd-4690-a7c6-77cca53d6f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:25.056773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac64737-efcd-4690-a7c6-77cca53d6f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:25.077861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac64737-efcd-4690-a7c6-77cca53d6f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:25.082492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac64737-efcd-4690-a7c6-77cca53d6f22 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:25.102689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1324/2626] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:27.686889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fd838-4c49-4ee3-838d-c2a683a499ae None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:27.688346 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-946fd838-4c49-4ee3-838d-c2a683a499ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:27.700965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fd838-4c49-4ee3-838d-c2a683a499ae None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:27.706954 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac3462-3591-4a70-9a4c-70fdb91f86be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:27.707592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac3462-3591-4a70-9a4c-70fdb91f86be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:27.708017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac3462-3591-4a70-9a4c-70fdb91f86be admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:27.708456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac3462-3591-4a70-9a4c-70fdb91f86be admin admin] PATH_INFO: `/v3/users/c205ee9f85924da79b870648698caf64` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:27.709584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac3462-3591-4a70-9a4c-70fdb91f86be admin admin] RBAC: Authorizing `identity:delete_user(user_id=c205ee9f85924da79b870648698caf64)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:27.711743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac3462-3591-4a70-9a4c-70fdb91f86be admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:27.892005 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3cac3462-3591-4a70-9a4c-70fdb91f86be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c205ee9f85924da79b870648698caf64'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:27.896109 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3cac3462-3591-4a70-9a4c-70fdb91f86be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c205ee9f85924da79b870648698caf64'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:27.996299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3cac3462-3591-4a70-9a4c-70fdb91f86be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c205ee9f85924da79b870648698caf64'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:28.007151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3cac3462-3591-4a70-9a4c-70fdb91f86be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c205ee9f85924da79b870648698caf64'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:28.008142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-3cac3462-3591-4a70-9a4c-70fdb91f86be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c205ee9f85924da79b870648698caf64'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:28.034674 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1303/2627] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:27 2022] DELETE /identity/v3/users/c205ee9f85924da79b870648698caf64 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:28.833025 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac64737-efcd-4690-a7c6-77cca53d6f22 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:28.845355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac64737-efcd-4690-a7c6-77cca53d6f22 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:28.941227 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c618a59-9522-4c32-86e0-fbe2a9c2baec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:28.941864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c618a59-9522-4c32-86e0-fbe2a9c2baec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:28.946746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c618a59-9522-4c32-86e0-fbe2a9c2baec admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:28.946746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c618a59-9522-4c32-86e0-fbe2a9c2baec admin admin] PATH_INFO: `/v3/projects/1a2ef5dbdbb34457963733b3507abb34` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:28.946746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c618a59-9522-4c32-86e0-fbe2a9c2baec admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a2ef5dbdbb34457963733b3507abb34)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:28.946746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c618a59-9522-4c32-86e0-fbe2a9c2baec admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:28.965584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:29.095900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1c618a59-9522-4c32-86e0-fbe2a9c2baec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a2ef5dbdbb34457963733b3507abb34', 'request_id': 'req-1c618a59-9522-4c32-86e0-fbe2a9c2baec'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:29.109857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1c618a59-9522-4c32-86e0-fbe2a9c2baec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a2ef5dbdbb34457963733b3507abb34', 'request_id': 'req-1c618a59-9522-4c32-86e0-fbe2a9c2baec'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:29.120807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1c618a59-9522-4c32-86e0-fbe2a9c2baec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a2ef5dbdbb34457963733b3507abb34', 'request_id': 'req-1c618a59-9522-4c32-86e0-fbe2a9c2baec'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:29.212111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1c618a59-9522-4c32-86e0-fbe2a9c2baec admin admin] The token cache is being invalidate because project 1a2ef5dbdbb34457963733b3507abb34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:29.213474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1c618a59-9522-4c32-86e0-fbe2a9c2baec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:29.215250 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-1c618a59-9522-4c32-86e0-fbe2a9c2baec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:29.222662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1325/2628] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:28 2022] DELETE /identity/v3/projects/1a2ef5dbdbb34457963733b3507abb34 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:29.224773 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a4890-8cca-4966-9d97-f2aaa1641eef None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:29.225245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a4890-8cca-4966-9d97-f2aaa1641eef None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:29.225507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a4890-8cca-4966-9d97-f2aaa1641eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:29.407179 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-a12a4890-8cca-4966-9d97-f2aaa1641eef None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:29.408245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a12a4890-8cca-4966-9d97-f2aaa1641eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:29.475802 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1304/2629] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:29.484413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca2186-b4f5-4697-b9cc-c56ea73fcf5d None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:29.484700 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca2186-b4f5-4697-b9cc-c56ea73fcf5d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:29.488799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca2186-b4f5-4697-b9cc-c56ea73fcf5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:29.505199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c1ca2186-b4f5-4697-b9cc-c56ea73fcf5d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:29.506428 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c1ca2186-b4f5-4697-b9cc-c56ea73fcf5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:29.519150 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1326/2630] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:29.523371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a4890-8cca-4966-9d97-f2aaa1641eef None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:29.534843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a4890-8cca-4966-9d97-f2aaa1641eef None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:29.540173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d998ff4-d4d3-4d1b-b27c-62bb8a2d14ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:29.541303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d998ff4-d4d3-4d1b-b27c-62bb8a2d14ba admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:29.541896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d998ff4-d4d3-4d1b-b27c-62bb8a2d14ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:29.542251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d998ff4-d4d3-4d1b-b27c-62bb8a2d14ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:29.543047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d998ff4-d4d3-4d1b-b27c-62bb8a2d14ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:50:29.543444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d998ff4-d4d3-4d1b-b27c-62bb8a2d14ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:29.546154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d998ff4-d4d3-4d1b-b27c-62bb8a2d14ba admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:29.568366 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1305/2631] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:50:29.569727 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca2186-b4f5-4697-b9cc-c56ea73fcf5d None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:29.582859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca2186-b4f5-4697-b9cc-c56ea73fcf5d None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:29.602861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956330c-38b0-4167-8ff0-a49256357f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:29.603491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956330c-38b0-4167-8ff0-a49256357f44 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:29.603766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956330c-38b0-4167-8ff0-a49256357f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:29.604349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b956330c-38b0-4167-8ff0-a49256357f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:29.605147 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956330c-38b0-4167-8ff0-a49256357f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:29.607381 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b956330c-38b0-4167-8ff0-a49256357f44 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:29.622291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d998ff4-d4d3-4d1b-b27c-62bb8a2d14ba admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:29.624610 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4d998ff4-d4d3-4d1b-b27c-62bb8a2d14ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:29.652167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d998ff4-d4d3-4d1b-b27c-62bb8a2d14ba admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:29.657937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11c7e5-c853-4d3f-8698-8679eb93bdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:29.658852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11c7e5-c853-4d3f-8698-8679eb93bdec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:29.659066 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11c7e5-c853-4d3f-8698-8679eb93bdec admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:29.659350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11c7e5-c853-4d3f-8698-8679eb93bdec admin admin] PATH_INFO: `/v3/users/7c623b10040f458a92aa9db4d96d8271` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:29.660484 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e11c7e5-c853-4d3f-8698-8679eb93bdec admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c623b10040f458a92aa9db4d96d8271)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:29.662943 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e11c7e5-c853-4d3f-8698-8679eb93bdec admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:29.688275 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:29.702928 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1327/2632] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:29 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:29.708458 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956330c-38b0-4167-8ff0-a49256357f44 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:29.783180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b956330c-38b0-4167-8ff0-a49256357f44 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:29.790026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a6766-ac77-4cc0-a522-90e6604765db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:29.790875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a6766-ac77-4cc0-a522-90e6604765db admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:29.791158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a6766-ac77-4cc0-a522-90e6604765db admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:29.791429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a6766-ac77-4cc0-a522-90e6604765db admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:29.792121 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a6766-ac77-4cc0-a522-90e6604765db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:29.794139 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04a6766-ac77-4cc0-a522-90e6604765db admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:29.840214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0e11c7e5-c853-4d3f-8698-8679eb93bdec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c623b10040f458a92aa9db4d96d8271'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:29.840777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0e11c7e5-c853-4d3f-8698-8679eb93bdec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c623b10040f458a92aa9db4d96d8271'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:29.882608 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1328/2633] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:29 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:29.884231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a6766-ac77-4cc0-a522-90e6604765db admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:29.893641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0e11c7e5-c853-4d3f-8698-8679eb93bdec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c623b10040f458a92aa9db4d96d8271'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:29.896511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a6766-ac77-4cc0-a522-90e6604765db admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:29.902326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0e11c7e5-c853-4d3f-8698-8679eb93bdec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c623b10040f458a92aa9db4d96d8271'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:29.902520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b737f-3136-4711-9ea6-3ac3152eeccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:29.902952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b737f-3136-4711-9ea6-3ac3152eeccd admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:29.903312 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b737f-3136-4711-9ea6-3ac3152eeccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:29.903570 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b737f-3136-4711-9ea6-3ac3152eeccd admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:29.904084 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0e11c7e5-c853-4d3f-8698-8679eb93bdec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c623b10040f458a92aa9db4d96d8271'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:29.904270 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b737f-3136-4711-9ea6-3ac3152eeccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:29.907202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b737f-3136-4711-9ea6-3ac3152eeccd admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:29.923987 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1329/2634] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:29.924935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b737f-3136-4711-9ea6-3ac3152eeccd admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:29.926724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c15b737f-3136-4711-9ea6-3ac3152eeccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:29.931941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1306/2635] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:29 2022] DELETE /identity/v3/users/7c623b10040f458a92aa9db4d96d8271 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:50:29.940547 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b737f-3136-4711-9ea6-3ac3152eeccd admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:30.001744 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:30.013332 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403d5d1-1aad-41b8-b68d-418a6f6ec92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:30.014803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403d5d1-1aad-41b8-b68d-418a6f6ec92b admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:30.015287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403d5d1-1aad-41b8-b68d-418a6f6ec92b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:30.017810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403d5d1-1aad-41b8-b68d-418a6f6ec92b admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:30.017810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403d5d1-1aad-41b8-b68d-418a6f6ec92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:30.018067 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403d5d1-1aad-41b8-b68d-418a6f6ec92b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:30.028682 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1330/2636] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:30.033625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11c7e5-c853-4d3f-8698-8679eb93bdec admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:30.043686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11c7e5-c853-4d3f-8698-8679eb93bdec admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:30.048364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff559c-97ac-4dae-8e6b-1d558c1bcede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:30.049026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff559c-97ac-4dae-8e6b-1d558c1bcede admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:30.049301 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff559c-97ac-4dae-8e6b-1d558c1bcede admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:30.049564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff559c-97ac-4dae-8e6b-1d558c1bcede admin admin] PATH_INFO: `/v3/projects/85c423a5bb55440da94b5cb326fab510/users/89868ef227934702b7cb60db77e1b026/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:30.064682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff559c-97ac-4dae-8e6b-1d558c1bcede admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c423a5bb55440da94b5cb326fab510, user_id=89868ef227934702b7cb60db77e1b026, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:30.068453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff559c-97ac-4dae-8e6b-1d558c1bcede admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:30.151756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1307/2637] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:30 2022] PUT /identity/v3/projects/85c423a5bb55440da94b5cb326fab510/users/89868ef227934702b7cb60db77e1b026/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:30.163647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab81b2b-c277-4ea8-ad44-4765958f59ae None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:30.163952 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab81b2b-c277-4ea8-ad44-4765958f59ae None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:30.164225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab81b2b-c277-4ea8-ad44-4765958f59ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:30.195046 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-cab81b2b-c277-4ea8-ad44-4765958f59ae None None] MFA Rules not processed for user `89868ef227934702b7cb60db77e1b026`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:30.196270 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-cab81b2b-c277-4ea8-ad44-4765958f59ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:30.224136 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff559c-97ac-4dae-8e6b-1d558c1bcede admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:30.226528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-36ff559c-97ac-4dae-8e6b-1d558c1bcede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:30.273706 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1331/2638] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:30.275012 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab81b2b-c277-4ea8-ad44-4765958f59ae None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:30.324045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff559c-97ac-4dae-8e6b-1d558c1bcede admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:30.329749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:30.330414 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:30.330691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:30.331233 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] PATH_INFO: `/v3/projects/8c494b8167314a3aacd699afa7e6a509` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:30.332079 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c494b8167314a3aacd699afa7e6a509)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:30.333994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:30.353438 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab81b2b-c277-4ea8-ad44-4765958f59ae None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:30.359277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb9a64-ca50-4ebd-94d7-95901816caaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:30.360288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb9a64-ca50-4ebd-94d7-95901816caaa admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:30.360557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb9a64-ca50-4ebd-94d7-95901816caaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:30.360837 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb9a64-ca50-4ebd-94d7-95901816caaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:30.361454 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb9a64-ca50-4ebd-94d7-95901816caaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:30.364048 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb9a64-ca50-4ebd-94d7-95901816caaa admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:30.478561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c494b8167314a3aacd699afa7e6a509', 'request_id': 'req-38434693-5337-419f-9d39-255f9e3db7f8'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:30.479170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1332/2639] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:30 2022] POST /identity/v3/projects => generated 373 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:30.484409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb9a64-ca50-4ebd-94d7-95901816caaa admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:30.488778 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c494b8167314a3aacd699afa7e6a509', 'request_id': 'req-38434693-5337-419f-9d39-255f9e3db7f8'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:30.495894 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c494b8167314a3aacd699afa7e6a509', 'request_id': 'req-38434693-5337-419f-9d39-255f9e3db7f8'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:30.559414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb9a64-ca50-4ebd-94d7-95901816caaa admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:30.565074 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19be932-44d5-4a9b-a5f2-94585b90d02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:30.565765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19be932-44d5-4a9b-a5f2-94585b90d02f admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:30.566139 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19be932-44d5-4a9b-a5f2-94585b90d02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:30.566816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19be932-44d5-4a9b-a5f2-94585b90d02f admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:30.567497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19be932-44d5-4a9b-a5f2-94585b90d02f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:30.569510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19be932-44d5-4a9b-a5f2-94585b90d02f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:30.579499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] The token cache is being invalidate because project 8c494b8167314a3aacd699afa7e6a509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:30.580019 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:30.581481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:30.585297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1308/2640] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:30 2022] DELETE /identity/v3/projects/8c494b8167314a3aacd699afa7e6a509 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:50:30.591173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:30.592866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:30.599278 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:30.660294 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1333/2641] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:30 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:30.664505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19be932-44d5-4a9b-a5f2-94585b90d02f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:30.666353 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d19be932-44d5-4a9b-a5f2-94585b90d02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:30.677373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38434693-5337-419f-9d39-255f9e3db7f8 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:30.682991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:30.684205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:30.684706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:30.685131 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] PATH_INFO: `/v3/users/49fb50eb99b64f588c0312c2935c82fa` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:30.685471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19be932-44d5-4a9b-a5f2-94585b90d02f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:30.687195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49fb50eb99b64f588c0312c2935c82fa)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:30.689367 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:30.694190 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2a76c-6b31-4ce3-b199-71cb09088fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:30.694807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2a76c-6b31-4ce3-b199-71cb09088fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:30.695084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2a76c-6b31-4ce3-b199-71cb09088fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:30.695364 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2a76c-6b31-4ce3-b199-71cb09088fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:30.696015 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2a76c-6b31-4ce3-b199-71cb09088fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:30.697964 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2a76c-6b31-4ce3-b199-71cb09088fc5 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:30.703190 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:30.717713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1334/2642] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:50:30.721453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2a76c-6b31-4ce3-b199-71cb09088fc5 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:30.731950 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2a76c-6b31-4ce3-b199-71cb09088fc5 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:30.738659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89aacc4-a868-430f-a854-d62325fa7a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:30.738939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89aacc4-a868-430f-a854-d62325fa7a1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:30.739426 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89aacc4-a868-430f-a854-d62325fa7a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:30.739825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89aacc4-a868-430f-a854-d62325fa7a1a admin admin] PATH_INFO: `/v3/projects/102f9ebeba8c4ffbaf577ead42ee6fcf/users/ce9d058992174c2aaf8f1b096a4cad53/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:30.756196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89aacc4-a868-430f-a854-d62325fa7a1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=102f9ebeba8c4ffbaf577ead42ee6fcf, user_id=ce9d058992174c2aaf8f1b096a4cad53, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:30.759167 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89aacc4-a868-430f-a854-d62325fa7a1a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:30.837863 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1335/2643] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:30 2022] PUT /identity/v3/projects/102f9ebeba8c4ffbaf577ead42ee6fcf/users/ce9d058992174c2aaf8f1b096a4cad53/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:30.841891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89aacc4-a868-430f-a854-d62325fa7a1a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:30.883847 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49fb50eb99b64f588c0312c2935c82fa'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:30.884676 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49fb50eb99b64f588c0312c2935c82fa'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:30.922573 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89aacc4-a868-430f-a854-d62325fa7a1a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:30.928663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7990d-1abc-471c-a375-5a9ebfdb223b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:30.929280 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7990d-1abc-471c-a375-5a9ebfdb223b admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:30.929727 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7990d-1abc-471c-a375-5a9ebfdb223b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:30.930057 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7990d-1abc-471c-a375-5a9ebfdb223b admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:30.930739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7990d-1abc-471c-a375-5a9ebfdb223b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:30.932341 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7990d-1abc-471c-a375-5a9ebfdb223b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:30.942748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1336/2644] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:30 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:30.947027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7990d-1abc-471c-a375-5a9ebfdb223b admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:30.969875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7990d-1abc-471c-a375-5a9ebfdb223b admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:30.978078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7de029-e8cd-44c6-95d1-24f73b4c3545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:30.978078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7de029-e8cd-44c6-95d1-24f73b4c3545 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:30.978078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7de029-e8cd-44c6-95d1-24f73b4c3545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:30.978078 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7de029-e8cd-44c6-95d1-24f73b4c3545 admin admin] PATH_INFO: `/v3/projects/102f9ebeba8c4ffbaf577ead42ee6fcf/users/ce9d058992174c2aaf8f1b096a4cad53/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:30.982662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7de029-e8cd-44c6-95d1-24f73b4c3545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=102f9ebeba8c4ffbaf577ead42ee6fcf, user_id=ce9d058992174c2aaf8f1b096a4cad53, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:30.988871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7de029-e8cd-44c6-95d1-24f73b4c3545 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:30.997637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49fb50eb99b64f588c0312c2935c82fa'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:31.008206 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49fb50eb99b64f588c0312c2935c82fa'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:31.009798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49fb50eb99b64f588c0312c2935c82fa'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:31.015188 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1337/2645] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:30 2022] PUT /identity/v3/projects/102f9ebeba8c4ffbaf577ead42ee6fcf/users/ce9d058992174c2aaf8f1b096a4cad53/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:31.017074 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ad7d4-11e4-47d2-9345-5122c68d3d21 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:31.017421 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ad7d4-11e4-47d2-9345-5122c68d3d21 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:31.017925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ad7d4-11e4-47d2-9345-5122c68d3d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:31.041597 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1309/2646] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:30 2022] DELETE /identity/v3/users/49fb50eb99b64f588c0312c2935c82fa => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:50:31.055942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-428ad7d4-11e4-47d2-9345-5122c68d3d21 None None] MFA Rules not processed for user `ce9d058992174c2aaf8f1b096a4cad53`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:31.057722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-428ad7d4-11e4-47d2-9345-5122c68d3d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:31.156516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1338/2647] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:31.170041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:31.176455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:31.261132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfef16-2a61-45b6-a54e-4267a607b251 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:31.267282 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11028b76-4337-4f17-a799-541ad8317ac6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:31.269878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11028b76-4337-4f17-a799-541ad8317ac6 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:31.269878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11028b76-4337-4f17-a799-541ad8317ac6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:31.269878 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11028b76-4337-4f17-a799-541ad8317ac6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:31.279482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11028b76-4337-4f17-a799-541ad8317ac6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:31.281981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11028b76-4337-4f17-a799-541ad8317ac6 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:31.292556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1310/2648] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:31.356214 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ad7d4-11e4-47d2-9345-5122c68d3d21 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:31.358731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-428ad7d4-11e4-47d2-9345-5122c68d3d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:31.450307 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ad7d4-11e4-47d2-9345-5122c68d3d21 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:31.456749 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:31.460973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb31cab-bece-4de7-963d-90af3bfcfee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:31.461860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb31cab-bece-4de7-963d-90af3bfcfee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:31.462139 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb31cab-bece-4de7-963d-90af3bfcfee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:31.462659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb31cab-bece-4de7-963d-90af3bfcfee3 admin admin] PATH_INFO: `/v3/projects/728e268ef22e470194545edea611eb32` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:31.464117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb31cab-bece-4de7-963d-90af3bfcfee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=728e268ef22e470194545edea611eb32)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:31.466123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb31cab-bece-4de7-963d-90af3bfcfee3 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:31.628419 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-adb31cab-bece-4de7-963d-90af3bfcfee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728e268ef22e470194545edea611eb32', 'request_id': 'req-adb31cab-bece-4de7-963d-90af3bfcfee3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:31.635090 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-adb31cab-bece-4de7-963d-90af3bfcfee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728e268ef22e470194545edea611eb32', 'request_id': 'req-adb31cab-bece-4de7-963d-90af3bfcfee3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:31.642265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-adb31cab-bece-4de7-963d-90af3bfcfee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728e268ef22e470194545edea611eb32', 'request_id': 'req-adb31cab-bece-4de7-963d-90af3bfcfee3'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:31.819975 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-adb31cab-bece-4de7-963d-90af3bfcfee3 admin admin] The token cache is being invalidate because project 728e268ef22e470194545edea611eb32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:31.820367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-adb31cab-bece-4de7-963d-90af3bfcfee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:31.821081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-adb31cab-bece-4de7-963d-90af3bfcfee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:31.822996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1339/2649] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:31 2022] DELETE /identity/v3/projects/728e268ef22e470194545edea611eb32 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:32.513018 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11028b76-4337-4f17-a799-541ad8317ac6 service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:32.515541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-11028b76-4337-4f17-a799-541ad8317ac6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:32.534919 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11028b76-4337-4f17-a799-541ad8317ac6 service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:32.558296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:32.606946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de2fcc-b06e-439e-9f7c-5ec9c6ed5541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:32.607933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de2fcc-b06e-439e-9f7c-5ec9c6ed5541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:32.608292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de2fcc-b06e-439e-9f7c-5ec9c6ed5541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:32.608768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de2fcc-b06e-439e-9f7c-5ec9c6ed5541 admin admin] PATH_INFO: `/v3/users/89868ef227934702b7cb60db77e1b026` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:32.609983 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de2fcc-b06e-439e-9f7c-5ec9c6ed5541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89868ef227934702b7cb60db77e1b026)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:32.611953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de2fcc-b06e-439e-9f7c-5ec9c6ed5541 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:32.664571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c031c0f-92ee-4d50-b75a-b12eaba92f98 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:32.665115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c031c0f-92ee-4d50-b75a-b12eaba92f98 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:32.665115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c031c0f-92ee-4d50-b75a-b12eaba92f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:32.695887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-9c031c0f-92ee-4d50-b75a-b12eaba92f98 None None] MFA Rules not processed for user `7092b595cffa4ba48d45982165ac27b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:32.697128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-9c031c0f-92ee-4d50-b75a-b12eaba92f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:32.767184 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1340/2650] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:50:32.771620 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c031c0f-92ee-4d50-b75a-b12eaba92f98 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:32.773111 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-9c031c0f-92ee-4d50-b75a-b12eaba92f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:32.788247 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c031c0f-92ee-4d50-b75a-b12eaba92f98 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:32.793053 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad87063-13f3-4609-b8cf-fe9c697a76c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:32.793740 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad87063-13f3-4609-b8cf-fe9c697a76c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:32.794136 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad87063-13f3-4609-b8cf-fe9c697a76c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:32.794470 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad87063-13f3-4609-b8cf-fe9c697a76c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:32.795179 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad87063-13f3-4609-b8cf-fe9c697a76c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:32.797415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad87063-13f3-4609-b8cf-fe9c697a76c8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:32.840024 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d8de2fcc-b06e-439e-9f7c-5ec9c6ed5541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89868ef227934702b7cb60db77e1b026'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:32.840666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d8de2fcc-b06e-439e-9f7c-5ec9c6ed5541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89868ef227934702b7cb60db77e1b026'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:32.881000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1341/2651] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:32 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:50:32.884634 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad87063-13f3-4609-b8cf-fe9c697a76c8 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:32.895021 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d8de2fcc-b06e-439e-9f7c-5ec9c6ed5541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89868ef227934702b7cb60db77e1b026'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:32.906747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d8de2fcc-b06e-439e-9f7c-5ec9c6ed5541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89868ef227934702b7cb60db77e1b026'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:32.907944 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d8de2fcc-b06e-439e-9f7c-5ec9c6ed5541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89868ef227934702b7cb60db77e1b026'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:32.914978 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:32.930161 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1311/2652] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:32 2022] DELETE /identity/v3/users/89868ef227934702b7cb60db77e1b026 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:32.934522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257cf9d-60cc-41f8-85bc-2e7fdd62531e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:32.934811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257cf9d-60cc-41f8-85bc-2e7fdd62531e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:32.935084 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257cf9d-60cc-41f8-85bc-2e7fdd62531e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:32.969855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-1257cf9d-60cc-41f8-85bc-2e7fdd62531e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:32.969855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1257cf9d-60cc-41f8-85bc-2e7fdd62531e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:32.979652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad87063-13f3-4609-b8cf-fe9c697a76c8 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:33.050399 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1312/2653] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:50:33.060151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257cf9d-60cc-41f8-85bc-2e7fdd62531e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:33.062385 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1257cf9d-60cc-41f8-85bc-2e7fdd62531e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:33.072129 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601e1b5-510f-4966-ba9b-28e288fd67d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:33.073783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601e1b5-510f-4966-ba9b-28e288fd67d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:33.075916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601e1b5-510f-4966-ba9b-28e288fd67d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:33.076245 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601e1b5-510f-4966-ba9b-28e288fd67d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:33.077031 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601e1b5-510f-4966-ba9b-28e288fd67d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:33.079161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5601e1b5-510f-4966-ba9b-28e288fd67d1 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:33.112871 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:33.124461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257cf9d-60cc-41f8-85bc-2e7fdd62531e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:33.128397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae1376-dd7d-415d-b139-ed4773755f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:33.129134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae1376-dd7d-415d-b139-ed4773755f8e service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:33.129410 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae1376-dd7d-415d-b139-ed4773755f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:33.129678 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae1376-dd7d-415d-b139-ed4773755f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:33.138968 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae1376-dd7d-415d-b139-ed4773755f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:33.140418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae1376-dd7d-415d-b139-ed4773755f8e service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:33.149839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1313/2654] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:50:33.166455 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1342/2655] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:32 2022] POST /identity/v3/users => generated 299 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:50:33.168986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae1376-dd7d-415d-b139-ed4773755f8e service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:33.170040 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-99ae1376-dd7d-415d-b139-ed4773755f8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:33.188664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae1376-dd7d-415d-b139-ed4773755f8e service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:33.193380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af85ab9-4b2b-40f1-b819-dcf3d50e179c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:33.194473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af85ab9-4b2b-40f1-b819-dcf3d50e179c admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:33.194473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af85ab9-4b2b-40f1-b819-dcf3d50e179c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:33.194957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af85ab9-4b2b-40f1-b819-dcf3d50e179c admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:33.195578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af85ab9-4b2b-40f1-b819-dcf3d50e179c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:33.197228 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af85ab9-4b2b-40f1-b819-dcf3d50e179c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:33.209101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1314/2656] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:33.213124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601e1b5-510f-4966-ba9b-28e288fd67d1 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:33.224574 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601e1b5-510f-4966-ba9b-28e288fd67d1 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:33.229461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cb9c5-9096-4f33-9c30-4f519314a486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:33.231168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cb9c5-9096-4f33-9c30-4f519314a486 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:33.231168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cb9c5-9096-4f33-9c30-4f519314a486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:33.231168 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cb9c5-9096-4f33-9c30-4f519314a486 admin admin] PATH_INFO: `/v3/projects/4105a8ce820c4deeb0a3d46d49d5a481/users/d6c7ae6eb7be41a6a3d1d0996d7fdb37/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:33.246592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cb9c5-9096-4f33-9c30-4f519314a486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4105a8ce820c4deeb0a3d46d49d5a481, user_id=d6c7ae6eb7be41a6a3d1d0996d7fdb37, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:33.249318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cb9c5-9096-4f33-9c30-4f519314a486 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:33.357663 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1343/2657] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:33 2022] PUT /identity/v3/projects/4105a8ce820c4deeb0a3d46d49d5a481/users/d6c7ae6eb7be41a6a3d1d0996d7fdb37/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:33.361863 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af85ab9-4b2b-40f1-b819-dcf3d50e179c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:33.371841 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af85ab9-4b2b-40f1-b819-dcf3d50e179c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:33.459299 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072fffc-8c34-4dd9-a154-f0ff50a79a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:33.460219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072fffc-8c34-4dd9-a154-f0ff50a79a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:33.460219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072fffc-8c34-4dd9-a154-f0ff50a79a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:33.460942 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072fffc-8c34-4dd9-a154-f0ff50a79a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:33.461585 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072fffc-8c34-4dd9-a154-f0ff50a79a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:33.463740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072fffc-8c34-4dd9-a154-f0ff50a79a90 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:33.474485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1315/2658] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:50:33.478411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cb9c5-9096-4f33-9c30-4f519314a486 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:33.489028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cb9c5-9096-4f33-9c30-4f519314a486 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:33.494018 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52921b95-2617-49b8-8f15-022e60a83566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:33.495174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52921b95-2617-49b8-8f15-022e60a83566 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:33.495445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52921b95-2617-49b8-8f15-022e60a83566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:33.495712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52921b95-2617-49b8-8f15-022e60a83566 admin admin] PATH_INFO: `/v3/projects/4105a8ce820c4deeb0a3d46d49d5a481/users/d6c7ae6eb7be41a6a3d1d0996d7fdb37/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:33.497610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52921b95-2617-49b8-8f15-022e60a83566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4105a8ce820c4deeb0a3d46d49d5a481, user_id=d6c7ae6eb7be41a6a3d1d0996d7fdb37, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:33.501126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52921b95-2617-49b8-8f15-022e60a83566 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:33.511570 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1344/2659] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:33 2022] PUT /identity/v3/projects/4105a8ce820c4deeb0a3d46d49d5a481/users/d6c7ae6eb7be41a6a3d1d0996d7fdb37/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:50:33.602037 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072fffc-8c34-4dd9-a154-f0ff50a79a90 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:33.602037 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0072fffc-8c34-4dd9-a154-f0ff50a79a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:33.627270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072fffc-8c34-4dd9-a154-f0ff50a79a90 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:33.697804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:33.698457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:33.698717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:33.698983 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] PATH_INFO: `/v3/projects/85c423a5bb55440da94b5cb326fab510` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:33.700117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85c423a5bb55440da94b5cb326fab510)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:33.702692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:33.813637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85c423a5bb55440da94b5cb326fab510', 'request_id': 'req-c34771e3-6503-4a30-a145-37963ac12db2'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:33.822157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85c423a5bb55440da94b5cb326fab510', 'request_id': 'req-c34771e3-6503-4a30-a145-37963ac12db2'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:33.828966 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85c423a5bb55440da94b5cb326fab510', 'request_id': 'req-c34771e3-6503-4a30-a145-37963ac12db2'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:33.940785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] The token cache is being invalidate because project 85c423a5bb55440da94b5cb326fab510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:33.946607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:33.946607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:33.946607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1316/2660] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:33 2022] DELETE /identity/v3/projects/85c423a5bb55440da94b5cb326fab510 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:33.948339 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52921b95-2617-49b8-8f15-022e60a83566 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:33.950449 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-52921b95-2617-49b8-8f15-022e60a83566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:33.967853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52921b95-2617-49b8-8f15-022e60a83566 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:34.043420 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160e7e3-59e9-4f66-874a-2276428f6e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:34.044257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160e7e3-59e9-4f66-874a-2276428f6e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:34.044800 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160e7e3-59e9-4f66-874a-2276428f6e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:34.045389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160e7e3-59e9-4f66-874a-2276428f6e40 admin admin] PATH_INFO: `/v3/users/ce9d058992174c2aaf8f1b096a4cad53` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:34.047271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160e7e3-59e9-4f66-874a-2276428f6e40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9d058992174c2aaf8f1b096a4cad53)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:34.049457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5160e7e3-59e9-4f66-874a-2276428f6e40 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:34.079734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:34.206242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5160e7e3-59e9-4f66-874a-2276428f6e40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9d058992174c2aaf8f1b096a4cad53'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:34.208069 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5160e7e3-59e9-4f66-874a-2276428f6e40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9d058992174c2aaf8f1b096a4cad53'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:34.315789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5160e7e3-59e9-4f66-874a-2276428f6e40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9d058992174c2aaf8f1b096a4cad53'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:34.316896 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5160e7e3-59e9-4f66-874a-2276428f6e40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9d058992174c2aaf8f1b096a4cad53'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:34.325836 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-5160e7e3-59e9-4f66-874a-2276428f6e40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9d058992174c2aaf8f1b096a4cad53'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:34.344256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1345/2661] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:33 2022] DELETE /identity/v3/users/ce9d058992174c2aaf8f1b096a4cad53 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:34.858857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:34.858857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:34.863858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34771e3-6503-4a30-a145-37963ac12db2 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:34.935121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab6cfb-3576-4db3-92c1-89f9c80e252c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:34.936259 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab6cfb-3576-4db3-92c1-89f9c80e252c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:34.936712 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab6cfb-3576-4db3-92c1-89f9c80e252c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:34.937113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab6cfb-3576-4db3-92c1-89f9c80e252c admin admin] PATH_INFO: `/v3/projects/102f9ebeba8c4ffbaf577ead42ee6fcf` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:34.938880 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab6cfb-3576-4db3-92c1-89f9c80e252c admin admin] RBAC: Authorizing `identity:delete_project(project_id=102f9ebeba8c4ffbaf577ead42ee6fcf)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:34.941543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab6cfb-3576-4db3-92c1-89f9c80e252c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:35.058181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cdab6cfb-3576-4db3-92c1-89f9c80e252c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '102f9ebeba8c4ffbaf577ead42ee6fcf', 'request_id': 'req-cdab6cfb-3576-4db3-92c1-89f9c80e252c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:35.065413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cdab6cfb-3576-4db3-92c1-89f9c80e252c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '102f9ebeba8c4ffbaf577ead42ee6fcf', 'request_id': 'req-cdab6cfb-3576-4db3-92c1-89f9c80e252c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:35.073297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cdab6cfb-3576-4db3-92c1-89f9c80e252c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '102f9ebeba8c4ffbaf577ead42ee6fcf', 'request_id': 'req-cdab6cfb-3576-4db3-92c1-89f9c80e252c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:35.174458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cdab6cfb-3576-4db3-92c1-89f9c80e252c admin admin] The token cache is being invalidate because project 102f9ebeba8c4ffbaf577ead42ee6fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:35.175137 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cdab6cfb-3576-4db3-92c1-89f9c80e252c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:35.176034 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-cdab6cfb-3576-4db3-92c1-89f9c80e252c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:35.183883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1317/2662] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:34 2022] DELETE /identity/v3/projects/102f9ebeba8c4ffbaf577ead42ee6fcf => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:35.185347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881ba8e-d4d6-44b6-9f34-f45bf2717c62 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:35.186362 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881ba8e-d4d6-44b6-9f34-f45bf2717c62 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:35.186471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881ba8e-d4d6-44b6-9f34-f45bf2717c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:35.216075 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-9881ba8e-d4d6-44b6-9f34-f45bf2717c62 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:35.218583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-9881ba8e-d4d6-44b6-9f34-f45bf2717c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:35.312769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1346/2663] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:35.325790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a04b4e-be1e-45a2-94f2-d07eb22361a2 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:35.328595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a04b4e-be1e-45a2-94f2-d07eb22361a2 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:35.330553 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a04b4e-be1e-45a2-94f2-d07eb22361a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:35.361718 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-18a04b4e-be1e-45a2-94f2-d07eb22361a2 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:35.363439 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-18a04b4e-be1e-45a2-94f2-d07eb22361a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:35.374327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1318/2664] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:35.379058 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881ba8e-d4d6-44b6-9f34-f45bf2717c62 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:35.394178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881ba8e-d4d6-44b6-9f34-f45bf2717c62 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:35.399206 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ad193-1566-47d8-9957-9edfc80c913f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:35.400062 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ad193-1566-47d8-9957-9edfc80c913f admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:35.400427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ad193-1566-47d8-9957-9edfc80c913f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:35.401123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ad193-1566-47d8-9957-9edfc80c913f admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:35.402189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ad193-1566-47d8-9957-9edfc80c913f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:50:35.403962 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ad193-1566-47d8-9957-9edfc80c913f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:35.405260 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ad193-1566-47d8-9957-9edfc80c913f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:35.429285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1347/2665] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:50:35.429810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a04b4e-be1e-45a2-94f2-d07eb22361a2 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:35.439546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a04b4e-be1e-45a2-94f2-d07eb22361a2 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:35.449297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9df8d-fe0f-4b3c-bf7e-ab8ca8c4bb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:35.450345 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9df8d-fe0f-4b3c-bf7e-ab8ca8c4bb68 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:35.450872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9df8d-fe0f-4b3c-bf7e-ab8ca8c4bb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:35.451364 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9df8d-fe0f-4b3c-bf7e-ab8ca8c4bb68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:35.453755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9df8d-fe0f-4b3c-bf7e-ab8ca8c4bb68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:35.456948 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9df8d-fe0f-4b3c-bf7e-ab8ca8c4bb68 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:35.560776 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1319/2666] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:35 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:35.565571 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ad193-1566-47d8-9957-9edfc80c913f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:35.578460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ad193-1566-47d8-9957-9edfc80c913f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:35.667425 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6489f4-5324-4336-b677-cf1ad07f2d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:35.671133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6489f4-5324-4336-b677-cf1ad07f2d2c admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:35.671133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6489f4-5324-4336-b677-cf1ad07f2d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:35.671133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6489f4-5324-4336-b677-cf1ad07f2d2c admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:35.671689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6489f4-5324-4336-b677-cf1ad07f2d2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:35.674022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6489f4-5324-4336-b677-cf1ad07f2d2c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:35.749809 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1348/2667] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:35 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:50:35.753758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9df8d-fe0f-4b3c-bf7e-ab8ca8c4bb68 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:35.769294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9df8d-fe0f-4b3c-bf7e-ab8ca8c4bb68 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:35.782157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1838ce-7b5a-48ff-80f1-8e9516ebd752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:35.783794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1838ce-7b5a-48ff-80f1-8e9516ebd752 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:35.784260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1838ce-7b5a-48ff-80f1-8e9516ebd752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:35.784834 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1838ce-7b5a-48ff-80f1-8e9516ebd752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:35.785722 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1838ce-7b5a-48ff-80f1-8e9516ebd752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:35.789729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1838ce-7b5a-48ff-80f1-8e9516ebd752 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:35.808531 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1320/2668] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:35.813805 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6489f4-5324-4336-b677-cf1ad07f2d2c admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:35.831689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6489f4-5324-4336-b677-cf1ad07f2d2c admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:35.838403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04047f9-9fb2-4388-904f-e3583c43fb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:35.839335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04047f9-9fb2-4388-904f-e3583c43fb9a admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:35.839831 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04047f9-9fb2-4388-904f-e3583c43fb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:35.840344 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04047f9-9fb2-4388-904f-e3583c43fb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:35.841327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04047f9-9fb2-4388-904f-e3583c43fb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:35.844086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04047f9-9fb2-4388-904f-e3583c43fb9a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:35.861636 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1349/2669] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:50:35.862668 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1838ce-7b5a-48ff-80f1-8e9516ebd752 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:35.876679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1838ce-7b5a-48ff-80f1-8e9516ebd752 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:35.889013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53fe41-400d-4c7e-8446-eb27ca0b1390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:35.890780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53fe41-400d-4c7e-8446-eb27ca0b1390 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:35.891338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53fe41-400d-4c7e-8446-eb27ca0b1390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:35.891813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53fe41-400d-4c7e-8446-eb27ca0b1390 admin admin] PATH_INFO: `/v3/projects/20826df7075642abac335f93bb2f14cc/users/0d573f6c94a1488f935653958fcba974/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:35.915915 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53fe41-400d-4c7e-8446-eb27ca0b1390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20826df7075642abac335f93bb2f14cc, user_id=0d573f6c94a1488f935653958fcba974, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:35.920007 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53fe41-400d-4c7e-8446-eb27ca0b1390 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:36.017173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1321/2670] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:35 2022] PUT /identity/v3/projects/20826df7075642abac335f93bb2f14cc/users/0d573f6c94a1488f935653958fcba974/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:50:36.028133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7db27d-d1ff-4ae9-ad2e-495e9db2ee2f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:36.028260 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7db27d-d1ff-4ae9-ad2e-495e9db2ee2f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:36.028555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7db27d-d1ff-4ae9-ad2e-495e9db2ee2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:36.037677 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:36.059381 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-2c7db27d-d1ff-4ae9-ad2e-495e9db2ee2f None None] MFA Rules not processed for user `0d573f6c94a1488f935653958fcba974`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:36.060589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-2c7db27d-d1ff-4ae9-ad2e-495e9db2ee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:36.175376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1350/2671] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:36.179956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53fe41-400d-4c7e-8446-eb27ca0b1390 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:36.226901 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:36.269602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53fe41-400d-4c7e-8446-eb27ca0b1390 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:36.276584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502cb37-31c8-478f-8381-d19f3bf47fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:36.277221 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502cb37-31c8-478f-8381-d19f3bf47fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:36.277495 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502cb37-31c8-478f-8381-d19f3bf47fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:36.277750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502cb37-31c8-478f-8381-d19f3bf47fe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:36.278424 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502cb37-31c8-478f-8381-d19f3bf47fe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:36.280385 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502cb37-31c8-478f-8381-d19f3bf47fe1 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:36.396704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1322/2672] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:36 2022] POST /identity/v3/projects => generated 385 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:36.401358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7db27d-d1ff-4ae9-ad2e-495e9db2ee2f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:36.420325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7db27d-d1ff-4ae9-ad2e-495e9db2ee2f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:36.504307 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f571738-6a61-492d-adcf-92d2a3854d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:36.504961 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f571738-6a61-492d-adcf-92d2a3854d00 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:36.505239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f571738-6a61-492d-adcf-92d2a3854d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:36.505506 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f571738-6a61-492d-adcf-92d2a3854d00 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:36.506235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f571738-6a61-492d-adcf-92d2a3854d00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:36.508375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f571738-6a61-492d-adcf-92d2a3854d00 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:36.598163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1351/2673] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:36 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:36.602113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502cb37-31c8-478f-8381-d19f3bf47fe1 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:36.613877 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502cb37-31c8-478f-8381-d19f3bf47fe1 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:36.621558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab2a90-241b-41c6-83b5-db4298e737ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:36.621558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2a90-241b-41c6-83b5-db4298e737ca admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:36.621558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2a90-241b-41c6-83b5-db4298e737ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:36.621558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab2a90-241b-41c6-83b5-db4298e737ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:36.621987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab2a90-241b-41c6-83b5-db4298e737ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:36.622574 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab2a90-241b-41c6-83b5-db4298e737ca admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:36.634527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1323/2674] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:36.638861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f571738-6a61-492d-adcf-92d2a3854d00 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:36.653682 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f571738-6a61-492d-adcf-92d2a3854d00 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:36.658990 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf262c-11ea-4fdd-8d91-82702f569a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:36.659608 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf262c-11ea-4fdd-8d91-82702f569a32 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:36.659886 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf262c-11ea-4fdd-8d91-82702f569a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:36.660414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf262c-11ea-4fdd-8d91-82702f569a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:36.660830 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf262c-11ea-4fdd-8d91-82702f569a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:36.662810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cf262c-11ea-4fdd-8d91-82702f569a32 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:36.673697 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1352/2675] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:36.678124 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab2a90-241b-41c6-83b5-db4298e737ca admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:36.689337 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab2a90-241b-41c6-83b5-db4298e737ca admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:36.695144 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d567c-a832-4095-8ff9-4a7c5bc93fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:36.695865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d567c-a832-4095-8ff9-4a7c5bc93fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:36.702485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d567c-a832-4095-8ff9-4a7c5bc93fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:36.702485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1d567c-a832-4095-8ff9-4a7c5bc93fb4 admin admin] PATH_INFO: `/v3/projects/7aa568d92e2649deb04d72297b4cc252/users/96a02a7d5c9b4fb39091b260ea434231/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:36.713157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d567c-a832-4095-8ff9-4a7c5bc93fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa568d92e2649deb04d72297b4cc252, user_id=96a02a7d5c9b4fb39091b260ea434231, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:36.715873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1d567c-a832-4095-8ff9-4a7c5bc93fb4 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:36.787250 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1324/2676] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:36 2022] PUT /identity/v3/projects/7aa568d92e2649deb04d72297b4cc252/users/96a02a7d5c9b4fb39091b260ea434231/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:36.798462 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb186f-227d-40d0-be71-1bad1df42ec2 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:36.798817 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb186f-227d-40d0-be71-1bad1df42ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:36.799095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb186f-227d-40d0-be71-1bad1df42ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:36.827500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-0beb186f-227d-40d0-be71-1bad1df42ec2 None None] MFA Rules not processed for user `96a02a7d5c9b4fb39091b260ea434231`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:36.828718 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0beb186f-227d-40d0-be71-1bad1df42ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:36.924733 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1353/2677] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:50:36.929996 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d567c-a832-4095-8ff9-4a7c5bc93fb4 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:37.055779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1d567c-a832-4095-8ff9-4a7c5bc93fb4 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:37.064750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7631453-6606-48ee-b9b6-87c8fc6d4925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:37.066226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7631453-6606-48ee-b9b6-87c8fc6d4925 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:37.066530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7631453-6606-48ee-b9b6-87c8fc6d4925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:37.066804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7631453-6606-48ee-b9b6-87c8fc6d4925 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:37.067447 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7631453-6606-48ee-b9b6-87c8fc6d4925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:37.071589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7631453-6606-48ee-b9b6-87c8fc6d4925 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:37.187934 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1325/2678] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:36 2022] POST /identity/v3/projects => generated 387 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:37.192024 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb186f-227d-40d0-be71-1bad1df42ec2 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:37.198447 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:37.214468 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb186f-227d-40d0-be71-1bad1df42ec2 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:37.306096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38071740-954c-41c4-ad8f-d03c90a327be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:37.308994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38071740-954c-41c4-ad8f-d03c90a327be admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:37.308994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38071740-954c-41c4-ad8f-d03c90a327be admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:37.309228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38071740-954c-41c4-ad8f-d03c90a327be admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:37.309799 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38071740-954c-41c4-ad8f-d03c90a327be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:37.312984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38071740-954c-41c4-ad8f-d03c90a327be admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:37.416234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1354/2679] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:37 2022] POST /identity/v3/users => generated 300 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:37.421050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7631453-6606-48ee-b9b6-87c8fc6d4925 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:37.432596 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7631453-6606-48ee-b9b6-87c8fc6d4925 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:37.438741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317381ad-55e8-4e23-be88-3693b7780606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:37.440093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317381ad-55e8-4e23-be88-3693b7780606 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:37.440093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317381ad-55e8-4e23-be88-3693b7780606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:37.440093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317381ad-55e8-4e23-be88-3693b7780606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:37.440536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317381ad-55e8-4e23-be88-3693b7780606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:37.442960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317381ad-55e8-4e23-be88-3693b7780606 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:37.455431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1326/2680] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:50:37.461251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38071740-954c-41c4-ad8f-d03c90a327be admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:37.475546 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38071740-954c-41c4-ad8f-d03c90a327be admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:37.481661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f472374-69dd-4066-981e-f212f8638380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:37.482445 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f472374-69dd-4066-981e-f212f8638380 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:37.482785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f472374-69dd-4066-981e-f212f8638380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:37.483642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f472374-69dd-4066-981e-f212f8638380 admin admin] PATH_INFO: `/v3/projects/98100653c5564dd7863e39b63463fa29/users/62b1213d1ea34566ac1f0f4363b4bfd1/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:37.504610 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f472374-69dd-4066-981e-f212f8638380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98100653c5564dd7863e39b63463fa29, user_id=62b1213d1ea34566ac1f0f4363b4bfd1, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:37.506350 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f472374-69dd-4066-981e-f212f8638380 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:37.573186 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1355/2681] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:37 2022] PUT /identity/v3/projects/98100653c5564dd7863e39b63463fa29/users/62b1213d1ea34566ac1f0f4363b4bfd1/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:37.577528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317381ad-55e8-4e23-be88-3693b7780606 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:37.588684 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317381ad-55e8-4e23-be88-3693b7780606 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:37.653263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:37.664652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c219d5-fe2e-4c53-9b29-0accd9c65a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:37.682158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c219d5-fe2e-4c53-9b29-0accd9c65a42 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:37.682530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c219d5-fe2e-4c53-9b29-0accd9c65a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:37.683452 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c219d5-fe2e-4c53-9b29-0accd9c65a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:37.684293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c219d5-fe2e-4c53-9b29-0accd9c65a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:37.686965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c219d5-fe2e-4c53-9b29-0accd9c65a42 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:37.699763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1327/2682] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:50:37.705375 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f472374-69dd-4066-981e-f212f8638380 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:37.718515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f472374-69dd-4066-981e-f212f8638380 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:37.724646 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e607d-5b4a-4c4d-9b7f-162efab528f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:37.725456 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e607d-5b4a-4c4d-9b7f-162efab528f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:37.725875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e607d-5b4a-4c4d-9b7f-162efab528f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:37.726623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e607d-5b4a-4c4d-9b7f-162efab528f8 admin admin] PATH_INFO: `/v3/projects/98100653c5564dd7863e39b63463fa29/users/62b1213d1ea34566ac1f0f4363b4bfd1/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:37.728410 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e607d-5b4a-4c4d-9b7f-162efab528f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98100653c5564dd7863e39b63463fa29, user_id=62b1213d1ea34566ac1f0f4363b4bfd1, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:37.731075 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e607d-5b4a-4c4d-9b7f-162efab528f8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:37.768462 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1356/2683] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:37 2022] PUT /identity/v3/projects/98100653c5564dd7863e39b63463fa29/users/62b1213d1ea34566ac1f0f4363b4bfd1/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:50:37.771087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182cdee-2d12-417a-b712-5887b3691ecd None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:37.771536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182cdee-2d12-417a-b712-5887b3691ecd None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:37.771956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182cdee-2d12-417a-b712-5887b3691ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:37.821224 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-3182cdee-2d12-417a-b712-5887b3691ecd None None] MFA Rules not processed for user `62b1213d1ea34566ac1f0f4363b4bfd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:37.822692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3182cdee-2d12-417a-b712-5887b3691ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:37.921657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1328/2684] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:50:37.934678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e607d-5b4a-4c4d-9b7f-162efab528f8 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:37.937904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-427e607d-5b4a-4c4d-9b7f-162efab528f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:38.015122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e607d-5b4a-4c4d-9b7f-162efab528f8 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:38.015122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa903ca-8229-49a0-958c-07e56be4bebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:38.015122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa903ca-8229-49a0-958c-07e56be4bebd service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:38.015122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa903ca-8229-49a0-958c-07e56be4bebd service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:38.015852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa903ca-8229-49a0-958c-07e56be4bebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:38.025557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa903ca-8229-49a0-958c-07e56be4bebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:38.026584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa903ca-8229-49a0-958c-07e56be4bebd service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:38.037720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1357/2685] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:39.244908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac03205-1579-4b45-a663-f75726b2a08d None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:39.245436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac03205-1579-4b45-a663-f75726b2a08d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:39.245436 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac03205-1579-4b45-a663-f75726b2a08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:39.250720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa903ca-8229-49a0-958c-07e56be4bebd service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:39.250720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-efa903ca-8229-49a0-958c-07e56be4bebd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:39.267243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa903ca-8229-49a0-958c-07e56be4bebd service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:39.271110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5c5b2-b881-4e0a-b8e0-594e9096ed6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:39.271734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5c5b2-b881-4e0a-b8e0-594e9096ed6d service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:39.272004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5c5b2-b881-4e0a-b8e0-594e9096ed6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:39.272267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5c5b2-b881-4e0a-b8e0-594e9096ed6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:39.290170 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-9ac03205-1579-4b45-a663-f75726b2a08d None None] MFA Rules not processed for user `d6c7ae6eb7be41a6a3d1d0996d7fdb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:39.291391 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9ac03205-1579-4b45-a663-f75726b2a08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:39.343316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:39.346748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5c5b2-b881-4e0a-b8e0-594e9096ed6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:39.348709 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5c5b2-b881-4e0a-b8e0-594e9096ed6d service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:39.364711 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1358/2686] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:39.384681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1329/2687] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:39.398740 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5c5b2-b881-4e0a-b8e0-594e9096ed6d service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:39.400176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c3c5c5b2-b881-4e0a-b8e0-594e9096ed6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:39.467657 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5c5b2-b881-4e0a-b8e0-594e9096ed6d service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:39.471399 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34a5e8-6907-48cd-b75e-fc539a3ac2bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d96ec6235843c5b67d87a3d0ff0f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:39.472006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34a5e8-6907-48cd-b75e-fc539a3ac2bd service glance] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:39.472277 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34a5e8-6907-48cd-b75e-fc539a3ac2bd service glance] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:39.472568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34a5e8-6907-48cd-b75e-fc539a3ac2bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:39.478877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ec34a5e8-6907-48cd-b75e-fc539a3ac2bd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:39.558655 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34a5e8-6907-48cd-b75e-fc539a3ac2bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:39.560084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34a5e8-6907-48cd-b75e-fc539a3ac2bd service glance] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:39.569648 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1359/2688] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:50:39.850832 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310bd68-cb8a-4ff7-8881-f49d4d142fda None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:39.851111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310bd68-cb8a-4ff7-8881-f49d4d142fda None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:39.851374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310bd68-cb8a-4ff7-8881-f49d4d142fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:39.871799 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-a310bd68-cb8a-4ff7-8881-f49d4d142fda None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:39.873002 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a310bd68-cb8a-4ff7-8881-f49d4d142fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:39.931098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:39.944617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1330/2689] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:50:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:40.131734 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34a5e8-6907-48cd-b75e-fc539a3ac2bd service glance] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:40.133312 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ec34a5e8-6907-48cd-b75e-fc539a3ac2bd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:40.227716 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34a5e8-6907-48cd-b75e-fc539a3ac2bd service glance] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:40.233177 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bfdb6-6760-4805-93a1-77bc0ca9d3cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:40.233795 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bfdb6-6760-4805-93a1-77bc0ca9d3cf service swift] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:40.234101 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bfdb6-6760-4805-93a1-77bc0ca9d3cf service swift] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:40.234366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bfdb6-6760-4805-93a1-77bc0ca9d3cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:40.243653 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bfdb6-6760-4805-93a1-77bc0ca9d3cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:40.245235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bfdb6-6760-4805-93a1-77bc0ca9d3cf service swift] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:40.255505 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1360/2690] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:50:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:50:44.683812 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310bd68-cb8a-4ff7-8881-f49d4d142fda None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:44.695129 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310bd68-cb8a-4ff7-8881-f49d4d142fda None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:44.699631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbe60e-398d-46cc-be10-95e5d4c67de0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:44.700240 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe60e-398d-46cc-be10-95e5d4c67de0 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:44.700515 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe60e-398d-46cc-be10-95e5d4c67de0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:44.700972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe60e-398d-46cc-be10-95e5d4c67de0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:44.783154 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbe60e-398d-46cc-be10-95e5d4c67de0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:44.784692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbe60e-398d-46cc-be10-95e5d4c67de0 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:44.796165 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1331/2691] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:49.117271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bfdb6-6760-4805-93a1-77bc0ca9d3cf service swift] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:49.166576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=100 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:49.190487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bfdb6-6760-4805-93a1-77bc0ca9d3cf service swift] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:49.195483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:49.196413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:49.196923 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:49.197529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] PATH_INFO: `/v3/users/0d573f6c94a1488f935653958fcba974` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:49.198864 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d573f6c94a1488f935653958fcba974)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:49.200969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:49.400143 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d573f6c94a1488f935653958fcba974'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:49.401328 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d573f6c94a1488f935653958fcba974'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:49.477115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d573f6c94a1488f935653958fcba974'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:49.477725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d573f6c94a1488f935653958fcba974'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:49.487970 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d573f6c94a1488f935653958fcba974'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:49.512769 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1361/2692] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:49 2022] DELETE /identity/v3/users/0d573f6c94a1488f935653958fcba974 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:49.515397 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3957e-8125-497c-80a3-b84a84b4289b None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:49.516207 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3957e-8125-497c-80a3-b84a84b4289b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:49.516476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3957e-8125-497c-80a3-b84a84b4289b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:49.539656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-1fd3957e-8125-497c-80a3-b84a84b4289b None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:49.540837 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1fd3957e-8125-497c-80a3-b84a84b4289b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:49.615259 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1332/2693] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:50:49.631617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:49.633422 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:49.701770 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56c88f-6c2b-4a8e-8e3c-7db7fc63753e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:49.705329 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6041e-1305-428b-b8e5-5f94664ac4fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:49.705966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6041e-1305-428b-b8e5-5f94664ac4fa service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:49.706247 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6041e-1305-428b-b8e5-5f94664ac4fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:49.706564 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6041e-1305-428b-b8e5-5f94664ac4fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:49.716058 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6041e-1305-428b-b8e5-5f94664ac4fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:49.717539 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6041e-1305-428b-b8e5-5f94664ac4fa service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:49.738595 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1362/2694] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:50:50.221934 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3957e-8125-497c-80a3-b84a84b4289b None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:50.223538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1fd3957e-8125-497c-80a3-b84a84b4289b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:50.237201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3957e-8125-497c-80a3-b84a84b4289b None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:50.247726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:50.254285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:50.254659 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:50.254827 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] PATH_INFO: `/v3/projects/20826df7075642abac335f93bb2f14cc` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:50.256247 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20826df7075642abac335f93bb2f14cc)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:50.258338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:50.449008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20826df7075642abac335f93bb2f14cc', 'request_id': 'req-bf478e93-1866-4153-83cb-82c4fc817b09'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:50.455138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20826df7075642abac335f93bb2f14cc', 'request_id': 'req-bf478e93-1866-4153-83cb-82c4fc817b09'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:50.461330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20826df7075642abac335f93bb2f14cc', 'request_id': 'req-bf478e93-1866-4153-83cb-82c4fc817b09'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:50.528294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] The token cache is being invalidate because project 20826df7075642abac335f93bb2f14cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:50.528902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:50.529076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:50.531214 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1333/2695] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:50 2022] DELETE /identity/v3/projects/20826df7075642abac335f93bb2f14cc => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:50.535246 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6041e-1305-428b-b8e5-5f94664ac4fa service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:50.545178 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6041e-1305-428b-b8e5-5f94664ac4fa service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:50.610363 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:50.611219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:50.611639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:50.611912 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] PATH_INFO: `/v3/users/96a02a7d5c9b4fb39091b260ea434231` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:50.613056 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=96a02a7d5c9b4fb39091b260ea434231)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:50.614995 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:50.617869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:50.624728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:50.626199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:50.650739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:50.728685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf478e93-1866-4153-83cb-82c4fc817b09 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:50.737284 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fee74-ca5a-4824-be2d-6a670f082e84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9975c0dab76f4ccbb4d2107e8e780ae5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:50.738276 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fee74-ca5a-4824-be2d-6a670f082e84 service placement] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:50.738715 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fee74-ca5a-4824-be2d-6a670f082e84 service placement] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:50.739133 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fee74-ca5a-4824-be2d-6a670f082e84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:50.741598 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-253fee74-ca5a-4824-be2d-6a670f082e84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:50.759784 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fee74-ca5a-4824-be2d-6a670f082e84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:50.762319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fee74-ca5a-4824-be2d-6a670f082e84 service placement] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:50.775804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1334/2696] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:50:50.855170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96a02a7d5c9b4fb39091b260ea434231'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:50.855925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96a02a7d5c9b4fb39091b260ea434231'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:50.905036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b0769-3a04-4df2-8702-f0f186a851e0 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:50.905319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b0769-3a04-4df2-8702-f0f186a851e0 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:50.905592 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b0769-3a04-4df2-8702-f0f186a851e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:50.926960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-6e9b0769-3a04-4df2-8702-f0f186a851e0 None None] MFA Rules not processed for user `7092b595cffa4ba48d45982165ac27b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:50.928432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6e9b0769-3a04-4df2-8702-f0f186a851e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:50.951194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96a02a7d5c9b4fb39091b260ea434231'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:50.952014 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96a02a7d5c9b4fb39091b260ea434231'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:50.959465 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96a02a7d5c9b4fb39091b260ea434231'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:50.978390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1363/2697] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:50 2022] DELETE /identity/v3/users/96a02a7d5c9b4fb39091b260ea434231 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:50.999749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1335/2698] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:50:51.517808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:51.519342 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:51.588759 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97995c-32da-4fba-a897-e4924aa491ef admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:51.593870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:51.594589 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:51.594858 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:51.595120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] PATH_INFO: `/v3/projects/7aa568d92e2649deb04d72297b4cc252` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:51.596238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa568d92e2649deb04d72297b4cc252)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:51.598134 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:51.706812 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa568d92e2649deb04d72297b4cc252', 'request_id': 'req-aa308568-c2a4-4e83-bfd3-873a67c1c2da'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:51.718731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa568d92e2649deb04d72297b4cc252', 'request_id': 'req-aa308568-c2a4-4e83-bfd3-873a67c1c2da'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:51.733677 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa568d92e2649deb04d72297b4cc252', 'request_id': 'req-aa308568-c2a4-4e83-bfd3-873a67c1c2da'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:51.803439 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] The token cache is being invalidate because project 7aa568d92e2649deb04d72297b4cc252 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:51.803822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:51.804578 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:51.806127 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1364/2699] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:51 2022] DELETE /identity/v3/projects/7aa568d92e2649deb04d72297b4cc252 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:51.810187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b0769-3a04-4df2-8702-f0f186a851e0 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:51.821898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b0769-3a04-4df2-8702-f0f186a851e0 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:51.877806 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221b5c0-ec6b-4ef3-a2bb-f193a42309a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:51.878458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221b5c0-ec6b-4ef3-a2bb-f193a42309a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:51.878726 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221b5c0-ec6b-4ef3-a2bb-f193a42309a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:51.878989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221b5c0-ec6b-4ef3-a2bb-f193a42309a0 admin admin] PATH_INFO: `/v3/users/62b1213d1ea34566ac1f0f4363b4bfd1` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:51.880090 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221b5c0-ec6b-4ef3-a2bb-f193a42309a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62b1213d1ea34566ac1f0f4363b4bfd1)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:51.882006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221b5c0-ec6b-4ef3-a2bb-f193a42309a0 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:52.065681 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:52.074068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-9221b5c0-ec6b-4ef3-a2bb-f193a42309a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62b1213d1ea34566ac1f0f4363b4bfd1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:52.074705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-9221b5c0-ec6b-4ef3-a2bb-f193a42309a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62b1213d1ea34566ac1f0f4363b4bfd1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:52.156262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-9221b5c0-ec6b-4ef3-a2bb-f193a42309a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62b1213d1ea34566ac1f0f4363b4bfd1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:52.164374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-9221b5c0-ec6b-4ef3-a2bb-f193a42309a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62b1213d1ea34566ac1f0f4363b4bfd1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:52.165153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-9221b5c0-ec6b-4ef3-a2bb-f193a42309a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62b1213d1ea34566ac1f0f4363b4bfd1'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:52.182688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1336/2700] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:51 2022] DELETE /identity/v3/users/62b1213d1ea34566ac1f0f4363b4bfd1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:53.035495 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:53.037584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:53.055249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa308568-c2a4-4e83-bfd3-873a67c1c2da admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:53.064274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:53.120915 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8932f8-fdb1-413a-922f-98f1df62888e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:53.122998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8932f8-fdb1-413a-922f-98f1df62888e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:53.122998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8932f8-fdb1-413a-922f-98f1df62888e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:53.122998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8932f8-fdb1-413a-922f-98f1df62888e admin admin] PATH_INFO: `/v3/projects/98100653c5564dd7863e39b63463fa29` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:53.124250 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8932f8-fdb1-413a-922f-98f1df62888e admin admin] RBAC: Authorizing `identity:delete_project(project_id=98100653c5564dd7863e39b63463fa29)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:53.126803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8932f8-fdb1-413a-922f-98f1df62888e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:53.264662 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6c8932f8-fdb1-413a-922f-98f1df62888e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98100653c5564dd7863e39b63463fa29', 'request_id': 'req-6c8932f8-fdb1-413a-922f-98f1df62888e'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:53.274746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6c8932f8-fdb1-413a-922f-98f1df62888e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98100653c5564dd7863e39b63463fa29', 'request_id': 'req-6c8932f8-fdb1-413a-922f-98f1df62888e'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:53.285012 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6c8932f8-fdb1-413a-922f-98f1df62888e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98100653c5564dd7863e39b63463fa29', 'request_id': 'req-6c8932f8-fdb1-413a-922f-98f1df62888e'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:53.374081 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6c8932f8-fdb1-413a-922f-98f1df62888e admin admin] The token cache is being invalidate because project 98100653c5564dd7863e39b63463fa29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:53.374365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6c8932f8-fdb1-413a-922f-98f1df62888e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:53.374985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-6c8932f8-fdb1-413a-922f-98f1df62888e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:53.376466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1365/2701] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:53 2022] DELETE /identity/v3/projects/98100653c5564dd7863e39b63463fa29 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:53.383571 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f1e6d-3239-4415-a4f1-9a2c9d3e2a08 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:53.384095 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f1e6d-3239-4415-a4f1-9a2c9d3e2a08 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:53.384217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f1e6d-3239-4415-a4f1-9a2c9d3e2a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:53.404296 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-e75f1e6d-3239-4415-a4f1-9a2c9d3e2a08 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:53.406245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-e75f1e6d-3239-4415-a4f1-9a2c9d3e2a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:53.466305 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1337/2702] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:53.475440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebe27e-5ea9-4765-9807-66dc2d58d73f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:53.475818 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebe27e-5ea9-4765-9807-66dc2d58d73f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:53.475976 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebe27e-5ea9-4765-9807-66dc2d58d73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:53.497295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-72ebe27e-5ea9-4765-9807-66dc2d58d73f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:53.498590 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-72ebe27e-5ea9-4765-9807-66dc2d58d73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:53.506472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1366/2703] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:53.510775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f1e6d-3239-4415-a4f1-9a2c9d3e2a08 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:53.521233 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f1e6d-3239-4415-a4f1-9a2c9d3e2a08 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:53.525891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6608b-72ea-4a63-947a-f33a12e8a27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:53.526440 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6608b-72ea-4a63-947a-f33a12e8a27e admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:53.526717 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6608b-72ea-4a63-947a-f33a12e8a27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:53.526986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6608b-72ea-4a63-947a-f33a12e8a27e admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:53.527747 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6608b-72ea-4a63-947a-f33a12e8a27e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:50:53.528050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6608b-72ea-4a63-947a-f33a12e8a27e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:53.530662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6608b-72ea-4a63-947a-f33a12e8a27e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:53.542621 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1338/2704] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:50:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:50:53.547140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebe27e-5ea9-4765-9807-66dc2d58d73f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:53.556235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebe27e-5ea9-4765-9807-66dc2d58d73f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:53.560903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a8c26-cb71-4d1e-acd2-581679510a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:53.561494 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a8c26-cb71-4d1e-acd2-581679510a39 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:53.561767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a8c26-cb71-4d1e-acd2-581679510a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:53.562065 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724a8c26-cb71-4d1e-acd2-581679510a39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:53.562690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a8c26-cb71-4d1e-acd2-581679510a39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:53.564566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724a8c26-cb71-4d1e-acd2-581679510a39 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:53.684120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1367/2705] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:53 2022] POST /identity/v3/projects => generated 369 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:53.689036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6608b-72ea-4a63-947a-f33a12e8a27e admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:53.699828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6608b-72ea-4a63-947a-f33a12e8a27e admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:53.770205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0554bf-aa30-4a2e-a870-5aa5058a8985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:53.771546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0554bf-aa30-4a2e-a870-5aa5058a8985 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:53.771828 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0554bf-aa30-4a2e-a870-5aa5058a8985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:53.772103 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0554bf-aa30-4a2e-a870-5aa5058a8985 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:53.772867 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0554bf-aa30-4a2e-a870-5aa5058a8985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:53.774833 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0554bf-aa30-4a2e-a870-5aa5058a8985 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:53.849455 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1339/2706] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:53 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:53.849562 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a8c26-cb71-4d1e-acd2-581679510a39 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:53.860827 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724a8c26-cb71-4d1e-acd2-581679510a39 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:53.869713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be2c9b-4184-403f-9bf5-d74100c3d0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:53.872289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be2c9b-4184-403f-9bf5-d74100c3d0df admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:53.872289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be2c9b-4184-403f-9bf5-d74100c3d0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:53.872289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be2c9b-4184-403f-9bf5-d74100c3d0df admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:53.872289 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be2c9b-4184-403f-9bf5-d74100c3d0df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:53.875002 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be2c9b-4184-403f-9bf5-d74100c3d0df admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:53.886361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1368/2707] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:50:53.891417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0554bf-aa30-4a2e-a870-5aa5058a8985 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:53.902068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0554bf-aa30-4a2e-a870-5aa5058a8985 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:53.907194 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeda313-3a9a-48cc-88c1-7ae36babf360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:53.907794 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeda313-3a9a-48cc-88c1-7ae36babf360 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:53.908077 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeda313-3a9a-48cc-88c1-7ae36babf360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:53.908344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeda313-3a9a-48cc-88c1-7ae36babf360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:53.908982 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeda313-3a9a-48cc-88c1-7ae36babf360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:53.910628 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeda313-3a9a-48cc-88c1-7ae36babf360 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:53.924658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1340/2708] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:53.928860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be2c9b-4184-403f-9bf5-d74100c3d0df admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:53.940089 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be2c9b-4184-403f-9bf5-d74100c3d0df admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:53.944687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7f538-da45-4a64-8ee3-c17e90808e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:53.945366 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7f538-da45-4a64-8ee3-c17e90808e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:53.945600 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7f538-da45-4a64-8ee3-c17e90808e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:53.945902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7f538-da45-4a64-8ee3-c17e90808e2d admin admin] PATH_INFO: `/v3/projects/6f196eb65bea4b319fad82efbac0d686/users/3f09e87adf664feb8315810427181c19/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:53.960696 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7f538-da45-4a64-8ee3-c17e90808e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f196eb65bea4b319fad82efbac0d686, user_id=3f09e87adf664feb8315810427181c19, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:53.963349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7f538-da45-4a64-8ee3-c17e90808e2d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:54.025074 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1369/2709] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:53 2022] PUT /identity/v3/projects/6f196eb65bea4b319fad82efbac0d686/users/3f09e87adf664feb8315810427181c19/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:54.037633 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847eec7-f946-4baa-90ad-960b83cdb939 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:54.037957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847eec7-f946-4baa-90ad-960b83cdb939 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:54.038255 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847eec7-f946-4baa-90ad-960b83cdb939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:54.065636 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-c847eec7-f946-4baa-90ad-960b83cdb939 None None] MFA Rules not processed for user `3f09e87adf664feb8315810427181c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:54.066792 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c847eec7-f946-4baa-90ad-960b83cdb939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:54.075595 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:54.131383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1341/2710] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:54.143424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7f538-da45-4a64-8ee3-c17e90808e2d admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:54.145196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7dc7f538-da45-4a64-8ee3-c17e90808e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:54.187092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:54.203236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7f538-da45-4a64-8ee3-c17e90808e2d admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:54.207095 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8604f853-36c3-4437-aa86-31499bc86d28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:54.207887 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604f853-36c3-4437-aa86-31499bc86d28 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:54.208408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604f853-36c3-4437-aa86-31499bc86d28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:54.209082 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8604f853-36c3-4437-aa86-31499bc86d28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:54.217897 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604f853-36c3-4437-aa86-31499bc86d28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:54.219308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8604f853-36c3-4437-aa86-31499bc86d28 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:54.227854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1370/2711] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:54.250167 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847eec7-f946-4baa-90ad-960b83cdb939 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:54.312089 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847eec7-f946-4baa-90ad-960b83cdb939 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:54.316742 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:54.317304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:54.317577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:54.317873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] PATH_INFO: `/v3/users/3f09e87adf664feb8315810427181c19` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:54.322479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f09e87adf664feb8315810427181c19)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:54.322479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:54.529255 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f09e87adf664feb8315810427181c19'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:54.530217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f09e87adf664feb8315810427181c19'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:54.596272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f09e87adf664feb8315810427181c19'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:54.603377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f09e87adf664feb8315810427181c19'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:54.603999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f09e87adf664feb8315810427181c19'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:54.620976 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1342/2712] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:50:54 2022] DELETE /identity/v3/users/3f09e87adf664feb8315810427181c19 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:54.626580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5d4fc-a9b4-4f70-9db5-9a922e4800c1 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:54.626861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5d4fc-a9b4-4f70-9db5-9a922e4800c1 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:54.627128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5d4fc-a9b4-4f70-9db5-9a922e4800c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:54.646544 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-63b5d4fc-a9b4-4f70-9db5-9a922e4800c1 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:54.647741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-63b5d4fc-a9b4-4f70-9db5-9a922e4800c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:54.705423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1371/2713] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:54.714967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:54.716338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:54.747198 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:54.772274 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddcef22-b0a3-4f81-8e25-a51eaa409bd2 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:54.775891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d26a1-506b-4083-9bae-15bbaa764fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:54.776513 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d26a1-506b-4083-9bae-15bbaa764fdc service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:54.776787 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d26a1-506b-4083-9bae-15bbaa764fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:54.777059 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d26a1-506b-4083-9bae-15bbaa764fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:54.786964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d26a1-506b-4083-9bae-15bbaa764fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:54.788689 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d26a1-506b-4083-9bae-15bbaa764fdc service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:54.799220 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1343/2714] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:50:55.238344 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5d4fc-a9b4-4f70-9db5-9a922e4800c1 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:55.239753 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-63b5d4fc-a9b4-4f70-9db5-9a922e4800c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:55.253690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5d4fc-a9b4-4f70-9db5-9a922e4800c1 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:55.259713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea9f32-787a-48a8-93fb-fb12c14e55f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:55.260250 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea9f32-787a-48a8-93fb-fb12c14e55f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:55.260520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea9f32-787a-48a8-93fb-fb12c14e55f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:55.260871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea9f32-787a-48a8-93fb-fb12c14e55f7 admin admin] PATH_INFO: `/v3/projects/6f196eb65bea4b319fad82efbac0d686` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:55.261899 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea9f32-787a-48a8-93fb-fb12c14e55f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f196eb65bea4b319fad82efbac0d686)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:55.264175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea9f32-787a-48a8-93fb-fb12c14e55f7 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:55.381689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-abea9f32-787a-48a8-93fb-fb12c14e55f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f196eb65bea4b319fad82efbac0d686', 'request_id': 'req-abea9f32-787a-48a8-93fb-fb12c14e55f7'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:55.387714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-abea9f32-787a-48a8-93fb-fb12c14e55f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f196eb65bea4b319fad82efbac0d686', 'request_id': 'req-abea9f32-787a-48a8-93fb-fb12c14e55f7'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:55.394429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-abea9f32-787a-48a8-93fb-fb12c14e55f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f196eb65bea4b319fad82efbac0d686', 'request_id': 'req-abea9f32-787a-48a8-93fb-fb12c14e55f7'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:55.494086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-abea9f32-787a-48a8-93fb-fb12c14e55f7 admin admin] The token cache is being invalidate because project 6f196eb65bea4b319fad82efbac0d686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:50:55.494291 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-abea9f32-787a-48a8-93fb-fb12c14e55f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:55.494910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-abea9f32-787a-48a8-93fb-fb12c14e55f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:50:55.496416 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1372/2715] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:50:55 2022] DELETE /identity/v3/projects/6f196eb65bea4b319fad82efbac0d686 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:50:55.502241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dc79f-bfee-4960-9e59-9598ec0862a3 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:55.502602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dc79f-bfee-4960-9e59-9598ec0862a3 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:55.502958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dc79f-bfee-4960-9e59-9598ec0862a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:55.523148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-ed2dc79f-bfee-4960-9e59-9598ec0862a3 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:55.524435 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-ed2dc79f-bfee-4960-9e59-9598ec0862a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:55.589082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1344/2716] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:55.599163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda06d3f-f5ed-4852-86a2-ca3318c7395c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:55.599444 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda06d3f-f5ed-4852-86a2-ca3318c7395c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:55.599713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda06d3f-f5ed-4852-86a2-ca3318c7395c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:55.620746 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-bda06d3f-f5ed-4852-86a2-ca3318c7395c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:55.622004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-bda06d3f-f5ed-4852-86a2-ca3318c7395c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:55.633906 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1373/2717] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:55.637152 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2dc79f-bfee-4960-9e59-9598ec0862a3 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:55.649949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2dc79f-bfee-4960-9e59-9598ec0862a3 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:55.659613 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6141e2-4167-429a-8d59-0472c9430150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:55.660242 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6141e2-4167-429a-8d59-0472c9430150 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:55.662746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6141e2-4167-429a-8d59-0472c9430150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:55.662746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6141e2-4167-429a-8d59-0472c9430150 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:55.662746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6141e2-4167-429a-8d59-0472c9430150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:50:55.662746 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6141e2-4167-429a-8d59-0472c9430150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:55.663639 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6141e2-4167-429a-8d59-0472c9430150 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:55.680347 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1345/2718] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:50:55.681118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda06d3f-f5ed-4852-86a2-ca3318c7395c None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:55.691299 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda06d3f-f5ed-4852-86a2-ca3318c7395c None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:55.696765 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471a60e-9ac3-4d93-921e-d7e576dc2efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:55.697377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471a60e-9ac3-4d93-921e-d7e576dc2efd admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:55.697640 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471a60e-9ac3-4d93-921e-d7e576dc2efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:55.697985 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471a60e-9ac3-4d93-921e-d7e576dc2efd admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:55.698581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471a60e-9ac3-4d93-921e-d7e576dc2efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:55.700500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471a60e-9ac3-4d93-921e-d7e576dc2efd admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:55.799436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1374/2719] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:50:55 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:55.803728 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6141e2-4167-429a-8d59-0472c9430150 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:55.815643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6141e2-4167-429a-8d59-0472c9430150 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:55.878476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803e0c4-6f40-4296-bfdc-feaf5bd0fbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:55.879083 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803e0c4-6f40-4296-bfdc-feaf5bd0fbdc admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:55.879356 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803e0c4-6f40-4296-bfdc-feaf5bd0fbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:55.879651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803e0c4-6f40-4296-bfdc-feaf5bd0fbdc admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:55.880249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803e0c4-6f40-4296-bfdc-feaf5bd0fbdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:55.882457 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803e0c4-6f40-4296-bfdc-feaf5bd0fbdc admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:55.980408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1346/2720] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:50:55 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:50:55.985377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471a60e-9ac3-4d93-921e-d7e576dc2efd admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:55.996197 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471a60e-9ac3-4d93-921e-d7e576dc2efd admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:56.001498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e48c2-cbeb-40b6-81b6-4b444932e5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:56.002167 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e48c2-cbeb-40b6-81b6-4b444932e5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:56.002436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e48c2-cbeb-40b6-81b6-4b444932e5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:56.002702 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e48c2-cbeb-40b6-81b6-4b444932e5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:56.003344 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e48c2-cbeb-40b6-81b6-4b444932e5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:56.005040 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e48c2-cbeb-40b6-81b6-4b444932e5f4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:56.015840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1375/2721] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:56.019964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803e0c4-6f40-4296-bfdc-feaf5bd0fbdc admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:56.031307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803e0c4-6f40-4296-bfdc-feaf5bd0fbdc admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:56.036093 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f8e65-97a9-4268-92d0-3496bf032840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:56.039306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f8e65-97a9-4268-92d0-3496bf032840 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:56.039306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f8e65-97a9-4268-92d0-3496bf032840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:56.039306 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f8e65-97a9-4268-92d0-3496bf032840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:56.039698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f8e65-97a9-4268-92d0-3496bf032840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:56.041427 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f8e65-97a9-4268-92d0-3496bf032840 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:56.053864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1347/2722] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:50:56.057705 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e48c2-cbeb-40b6-81b6-4b444932e5f4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:56.073464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e48c2-cbeb-40b6-81b6-4b444932e5f4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:56.088549 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124368c-700a-450a-85c9-b944920cf6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:56.089402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124368c-700a-450a-85c9-b944920cf6aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:56.089866 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124368c-700a-450a-85c9-b944920cf6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:56.090307 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124368c-700a-450a-85c9-b944920cf6aa admin admin] PATH_INFO: `/v3/projects/09daa04b5df3418ab5552d9e53521d7c/users/7ef05c0bdb7146bab49bea9057971cf6/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:56.113451 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124368c-700a-450a-85c9-b944920cf6aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=09daa04b5df3418ab5552d9e53521d7c, user_id=7ef05c0bdb7146bab49bea9057971cf6, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:56.116390 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124368c-700a-450a-85c9-b944920cf6aa admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:56.221408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:56.227301 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1376/2723] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:50:56 2022] PUT /identity/v3/projects/09daa04b5df3418ab5552d9e53521d7c/users/7ef05c0bdb7146bab49bea9057971cf6/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:50:56.233326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d747fc6-c7b8-4e97-99a8-e7e8a9ee1922 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:56.233602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d747fc6-c7b8-4e97-99a8-e7e8a9ee1922 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:56.233947 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d747fc6-c7b8-4e97-99a8-e7e8a9ee1922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:56.266137 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b1baf-1b51-49df-9cda-b65e545a789e None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:56.271592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b1baf-1b51-49df-9cda-b65e545a789e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:56.271592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b1baf-1b51-49df-9cda-b65e545a789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:56.285382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-3d747fc6-c7b8-4e97-99a8-e7e8a9ee1922 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:56.287767 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3d747fc6-c7b8-4e97-99a8-e7e8a9ee1922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:56.299372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-0b3b1baf-1b51-49df-9cda-b65e545a789e None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:56.300661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0b3b1baf-1b51-49df-9cda-b65e545a789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:56.331323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:50:56.376241 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1348/2724] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:56.381213 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1377/2725] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:50:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:50:56.387951 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d747fc6-c7b8-4e97-99a8-e7e8a9ee1922 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:56.389498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b1baf-1b51-49df-9cda-b65e545a789e None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:56.391140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3d747fc6-c7b8-4e97-99a8-e7e8a9ee1922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:56.392006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-0b3b1baf-1b51-49df-9cda-b65e545a789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:56.501361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d747fc6-c7b8-4e97-99a8-e7e8a9ee1922 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:56.502761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b1baf-1b51-49df-9cda-b65e545a789e None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:56.509650 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3dfc6-0fad-44bb-85f7-c1666c546df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:56.510834 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3dfc6-0fad-44bb-85f7-c1666c546df9 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:56.511108 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3dfc6-0fad-44bb-85f7-c1666c546df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:56.511378 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3dfc6-0fad-44bb-85f7-c1666c546df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:56.514940 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29fd5e-99de-47f0-9531-2675a02b0dae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:56.515560 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29fd5e-99de-47f0-9531-2675a02b0dae service swift] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:56.515920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29fd5e-99de-47f0-9531-2675a02b0dae service swift] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:56.516197 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29fd5e-99de-47f0-9531-2675a02b0dae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:56.534657 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29fd5e-99de-47f0-9531-2675a02b0dae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:56.535740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3dfc6-0fad-44bb-85f7-c1666c546df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:56.537659 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3dfc6-0fad-44bb-85f7-c1666c546df9 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:56.538925 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea29fd5e-99de-47f0-9531-2675a02b0dae service swift] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:56.564325 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1378/2726] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:50:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 16 05:50:56.564517 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1349/2727] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:50:58.766548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70623b31-5a8f-4f4e-a14e-829ec172cf48 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:58.767717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70623b31-5a8f-4f4e-a14e-829ec172cf48 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:58.768219 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70623b31-5a8f-4f4e-a14e-829ec172cf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:58.819802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-70623b31-5a8f-4f4e-a14e-829ec172cf48 None None] MFA Rules not processed for user `7ef05c0bdb7146bab49bea9057971cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:50:58.821978 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-70623b31-5a8f-4f4e-a14e-829ec172cf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:58.960128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1379/2728] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:50:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:50:58.972682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3dfc6-0fad-44bb-85f7-c1666c546df9 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:50:58.974385 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0ec3dfc6-0fad-44bb-85f7-c1666c546df9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:50:59.080012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3dfc6-0fad-44bb-85f7-c1666c546df9 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:50:59.085763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd4984-6f80-4f51-b725-0e89883b082d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:50:59.085763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd4984-6f80-4f51-b725-0e89883b082d service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:50:59.086449 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd4984-6f80-4f51-b725-0e89883b082d service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:50:59.087266 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd4984-6f80-4f51-b725-0e89883b082d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:50:59.098464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd4984-6f80-4f51-b725-0e89883b082d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:50:59.100443 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd4984-6f80-4f51-b725-0e89883b082d service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:50:59.118438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1350/2729] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:50:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:51:00.917457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a14c-0462-4ead-839a-2cbd5d39d4ff None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:51:00.918209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a14c-0462-4ead-839a-2cbd5d39d4ff None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:51:00.918209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a14c-0462-4ead-839a-2cbd5d39d4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:51:00.938264 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a3c1a14c-0462-4ead-839a-2cbd5d39d4ff None None] MFA Rules not processed for user `7ef05c0bdb7146bab49bea9057971cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:51:00.939492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a3c1a14c-0462-4ead-839a-2cbd5d39d4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:51:00.957428 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1380/2730] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:51:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:51:09.107286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=62 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:51:10.932027 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=92 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:51:18.472588 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf59f9c-350f-4573-9b85-3a6ed9834b1a None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:51:18.473113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf59f9c-350f-4573-9b85-3a6ed9834b1a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:51:18.473113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf59f9c-350f-4573-9b85-3a6ed9834b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:51:18.492760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0cf59f9c-350f-4573-9b85-3a6ed9834b1a None None] MFA Rules not processed for user `2dcce7be8c974a75a6ac82845865006f`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:51:18.493972 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0cf59f9c-350f-4573-9b85-3a6ed9834b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:51:18.551020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1351/2731] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:51:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:51:25.242285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573d067-6f85-4185-91f8-ca65a223892c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:51:25.242739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573d067-6f85-4185-91f8-ca65a223892c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:51:25.242854 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573d067-6f85-4185-91f8-ca65a223892c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:51:25.262026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-f573d067-6f85-4185-91f8-ca65a223892c None None] MFA Rules not processed for user `7ef05c0bdb7146bab49bea9057971cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:51:25.263225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f573d067-6f85-4185-91f8-ca65a223892c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:51:25.271100 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1381/2732] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:51:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:51:27.973235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf59f9c-350f-4573-9b85-3a6ed9834b1a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:51:27.974819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0cf59f9c-350f-4573-9b85-3a6ed9834b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:51:27.987475 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf59f9c-350f-4573-9b85-3a6ed9834b1a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:51:27.991102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed4cdb-993a-4f94-b530-bb45735c34cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:51:27.991769 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed4cdb-993a-4f94-b530-bb45735c34cb service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:51:27.992039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed4cdb-993a-4f94-b530-bb45735c34cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:51:27.992301 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed4cdb-993a-4f94-b530-bb45735c34cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:51:27.994027 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-60ed4cdb-993a-4f94-b530-bb45735c34cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:51:28.055401 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed4cdb-993a-4f94-b530-bb45735c34cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:51:28.056733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed4cdb-993a-4f94-b530-bb45735c34cb service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:51:28.066831 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1352/2733] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:51:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:51:35.256568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:51:38.061906 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=55 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:51:47.854117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400253-39f6-4c48-be77-591b2b8c8d4f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:51:47.854611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400253-39f6-4c48-be77-591b2b8c8d4f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:51:47.854717 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400253-39f6-4c48-be77-591b2b8c8d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:51:47.873922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a5400253-39f6-4c48-be77-591b2b8c8d4f None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:51:47.875161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a5400253-39f6-4c48-be77-591b2b8c8d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:51:47.882586 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1382/2734] 10.222.0.49 () {58 vars in 1009 bytes} [Fri Sep 16 05:51:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:51:47.906496 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed4cdb-993a-4f94-b530-bb45735c34cb service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:51:47.918371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed4cdb-993a-4f94-b530-bb45735c34cb service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:51:47.923929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87546dc-ea93-4080-97fe-ca43fe28c862 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:51:47.924052 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87546dc-ea93-4080-97fe-ca43fe28c862 service swift] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:51:47.924222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87546dc-ea93-4080-97fe-ca43fe28c862 service swift] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:51:47.924480 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87546dc-ea93-4080-97fe-ca43fe28c862 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:51:47.932642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87546dc-ea93-4080-97fe-ca43fe28c862 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:51:47.934432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87546dc-ea93-4080-97fe-ca43fe28c862 service swift] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:51:47.943210 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1353/2735] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:51:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:51:57.868440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:51:57.939199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=6 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:52:04.470142 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963cacf-2a70-458a-bfec-018da10e3dd9 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:04.470627 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963cacf-2a70-458a-bfec-018da10e3dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:04.470737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963cacf-2a70-458a-bfec-018da10e3dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:04.491611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c963cacf-2a70-458a-bfec-018da10e3dd9 None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:52:04.492801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c963cacf-2a70-458a-bfec-018da10e3dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:04.500248 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1383/2736] 10.222.0.49 () {58 vars in 967 bytes} [Fri Sep 16 05:52:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:52:04.512882 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87546dc-ea93-4080-97fe-ca43fe28c862 service swift] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:04.523733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87546dc-ea93-4080-97fe-ca43fe28c862 service swift] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:04.528896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a079d5-30f7-4d86-910d-b5f9d94988a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:04.529492 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a079d5-30f7-4d86-910d-b5f9d94988a0 service swift] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:04.529762 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a079d5-30f7-4d86-910d-b5f9d94988a0 service swift] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:04.530156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a079d5-30f7-4d86-910d-b5f9d94988a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:04.539385 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a079d5-30f7-4d86-910d-b5f9d94988a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:04.540973 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a079d5-30f7-4d86-910d-b5f9d94988a0 service swift] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:04.549121 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1354/2737] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:52:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:52:07.273665 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963cacf-2a70-458a-bfec-018da10e3dd9 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:07.275197 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c963cacf-2a70-458a-bfec-018da10e3dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:07.347229 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963cacf-2a70-458a-bfec-018da10e3dd9 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:07.352257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7991b3c-9bcf-4fef-a95e-0d5a2a66a45e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9975c0dab76f4ccbb4d2107e8e780ae5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:07.352857 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7991b3c-9bcf-4fef-a95e-0d5a2a66a45e service placement] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:07.353133 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7991b3c-9bcf-4fef-a95e-0d5a2a66a45e service placement] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:07.353403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7991b3c-9bcf-4fef-a95e-0d5a2a66a45e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:07.355124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d7991b3c-9bcf-4fef-a95e-0d5a2a66a45e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:07.367024 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7991b3c-9bcf-4fef-a95e-0d5a2a66a45e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:07.368687 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7991b3c-9bcf-4fef-a95e-0d5a2a66a45e service placement] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:07.378761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1384/2738] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:52:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:52:07.408290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a079d5-30f7-4d86-910d-b5f9d94988a0 service swift] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:07.420488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a079d5-30f7-4d86-910d-b5f9d94988a0 service swift] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:07.426643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df85840-7ca8-4680-aebf-b55fe7e427bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:07.427263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df85840-7ca8-4680-aebf-b55fe7e427bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:07.427535 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df85840-7ca8-4680-aebf-b55fe7e427bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:07.427798 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df85840-7ca8-4680-aebf-b55fe7e427bc admin admin] PATH_INFO: `/v3/users/7ef05c0bdb7146bab49bea9057971cf6` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:07.428957 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df85840-7ca8-4680-aebf-b55fe7e427bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ef05c0bdb7146bab49bea9057971cf6)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:07.435714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df85840-7ca8-4680-aebf-b55fe7e427bc admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:07.625835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0df85840-7ca8-4680-aebf-b55fe7e427bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ef05c0bdb7146bab49bea9057971cf6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:07.626511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0df85840-7ca8-4680-aebf-b55fe7e427bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ef05c0bdb7146bab49bea9057971cf6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:07.687219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0df85840-7ca8-4680-aebf-b55fe7e427bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ef05c0bdb7146bab49bea9057971cf6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:07.695460 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0df85840-7ca8-4680-aebf-b55fe7e427bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ef05c0bdb7146bab49bea9057971cf6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:07.696119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-0df85840-7ca8-4680-aebf-b55fe7e427bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ef05c0bdb7146bab49bea9057971cf6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:07.714168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1355/2739] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:52:07 2022] DELETE /identity/v3/users/7ef05c0bdb7146bab49bea9057971cf6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:52:07.957354 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7991b3c-9bcf-4fef-a95e-0d5a2a66a45e service placement] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:07.968566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7991b3c-9bcf-4fef-a95e-0d5a2a66a45e service placement] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:08.026579 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:08.027128 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:08.027403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:08.027672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8 admin admin] PATH_INFO: `/v3/projects/09daa04b5df3418ab5552d9e53521d7c` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:08.028836 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09daa04b5df3418ab5552d9e53521d7c)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:08.030764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:08.124407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09daa04b5df3418ab5552d9e53521d7c', 'request_id': 'req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:08.131370 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09daa04b5df3418ab5552d9e53521d7c', 'request_id': 'req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:08.138460 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09daa04b5df3418ab5552d9e53521d7c', 'request_id': 'req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:08.207675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8 admin admin] The token cache is being invalidate because project 09daa04b5df3418ab5552d9e53521d7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:52:08.207957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:08.208581 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-cbdbfd1f-09b0-4dce-965e-47e4633a48e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:08.210656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1385/2740] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:52:07 2022] DELETE /identity/v3/projects/09daa04b5df3418ab5552d9e53521d7c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:52:08.224265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811cd22-c329-42f2-9071-c43fc6f4e2a1 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:08.224265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811cd22-c329-42f2-9071-c43fc6f4e2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:08.224265 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811cd22-c329-42f2-9071-c43fc6f4e2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:08.236226 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0811cd22-c329-42f2-9071-c43fc6f4e2a1 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:52:08.238322 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0811cd22-c329-42f2-9071-c43fc6f4e2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:08.301063 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1356/2741] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:52:08.308772 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d7e74-e6a7-4351-abec-ef2d936781bb None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:08.309269 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d7e74-e6a7-4351-abec-ef2d936781bb None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:08.314338 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74d7e74-e6a7-4351-abec-ef2d936781bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:08.330032 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-c74d7e74-e6a7-4351-abec-ef2d936781bb None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:52:08.331920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c74d7e74-e6a7-4351-abec-ef2d936781bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:08.339902 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1386/2742] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:52:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:52:08.344130 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811cd22-c329-42f2-9071-c43fc6f4e2a1 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:08.354004 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811cd22-c329-42f2-9071-c43fc6f4e2a1 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:08.358706 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85362ff-c67d-4799-b3fb-03c595920e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:08.359302 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85362ff-c67d-4799-b3fb-03c595920e1b admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:08.359618 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85362ff-c67d-4799-b3fb-03c595920e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:08.361872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85362ff-c67d-4799-b3fb-03c595920e1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:08.361872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85362ff-c67d-4799-b3fb-03c595920e1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:52:08.361872 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85362ff-c67d-4799-b3fb-03c595920e1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:08.362496 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85362ff-c67d-4799-b3fb-03c595920e1b admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:08.374174 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1357/2743] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:52:08.379908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d7e74-e6a7-4351-abec-ef2d936781bb None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:08.389919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74d7e74-e6a7-4351-abec-ef2d936781bb None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:08.395618 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdbdf3-38ef-45d6-9d2c-2b7e33e09ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:08.396238 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdbdf3-38ef-45d6-9d2c-2b7e33e09ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:08.396515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdbdf3-38ef-45d6-9d2c-2b7e33e09ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:08.396783 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdbdf3-38ef-45d6-9d2c-2b7e33e09ff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:08.397423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cdbdf3-38ef-45d6-9d2c-2b7e33e09ff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:08.399361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cdbdf3-38ef-45d6-9d2c-2b7e33e09ff4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:08.404004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:52:08.476782 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1387/2744] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:52:08 2022] POST /identity/v3/projects => generated 399 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:52:08.481047 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85362ff-c67d-4799-b3fb-03c595920e1b admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:08.491374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85362ff-c67d-4799-b3fb-03c595920e1b admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:08.535119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:52:08.547623 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7075d8-53a2-46b7-a37d-e14904c50ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:08.548392 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7075d8-53a2-46b7-a37d-e14904c50ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:08.548666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7075d8-53a2-46b7-a37d-e14904c50ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:08.548931 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7075d8-53a2-46b7-a37d-e14904c50ed0 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:08.550536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7075d8-53a2-46b7-a37d-e14904c50ed0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:08.552464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7075d8-53a2-46b7-a37d-e14904c50ed0 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:08.629696 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1358/2745] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:52:08 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:52:08.633846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdbdf3-38ef-45d6-9d2c-2b7e33e09ff4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:08.644748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdbdf3-38ef-45d6-9d2c-2b7e33e09ff4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:08.649385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c0caa-c779-4a77-8364-1b2f8a634a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:08.650013 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c0caa-c779-4a77-8364-1b2f8a634a69 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:08.650285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c0caa-c779-4a77-8364-1b2f8a634a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:08.650551 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c0caa-c779-4a77-8364-1b2f8a634a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:08.651177 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c0caa-c779-4a77-8364-1b2f8a634a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:08.652847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c0caa-c779-4a77-8364-1b2f8a634a69 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:08.663361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1388/2746] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:52:08.667404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7075d8-53a2-46b7-a37d-e14904c50ed0 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:08.677189 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7075d8-53a2-46b7-a37d-e14904c50ed0 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:08.681672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125fd63-3279-4683-a621-ddc1bf990657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:08.682281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125fd63-3279-4683-a621-ddc1bf990657 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:08.682549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125fd63-3279-4683-a621-ddc1bf990657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:08.682809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125fd63-3279-4683-a621-ddc1bf990657 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:08.683434 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125fd63-3279-4683-a621-ddc1bf990657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:08.684994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125fd63-3279-4683-a621-ddc1bf990657 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:08.695402 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1359/2747] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:52:08.700951 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c0caa-c779-4a77-8364-1b2f8a634a69 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:08.715682 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c0caa-c779-4a77-8364-1b2f8a634a69 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:08.720823 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631e260-1547-4a42-a5b5-df77fdc9df0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:08.722236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631e260-1547-4a42-a5b5-df77fdc9df0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:08.722517 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631e260-1547-4a42-a5b5-df77fdc9df0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:08.722789 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631e260-1547-4a42-a5b5-df77fdc9df0d admin admin] PATH_INFO: `/v3/projects/994a2a4d9e704104a661c0392254223f/users/b9625ffd62304f9b943ea5325ea253f4/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:08.737633 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631e260-1547-4a42-a5b5-df77fdc9df0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=994a2a4d9e704104a661c0392254223f, user_id=b9625ffd62304f9b943ea5325ea253f4, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:08.740295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631e260-1547-4a42-a5b5-df77fdc9df0d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:08.801039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1389/2748] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:52:08 2022] PUT /identity/v3/projects/994a2a4d9e704104a661c0392254223f/users/b9625ffd62304f9b943ea5325ea253f4/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:52:08.865924 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600e15b-fb65-4fc0-b178-93454208c735 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:08.866201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600e15b-fb65-4fc0-b178-93454208c735 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:08.866469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600e15b-fb65-4fc0-b178-93454208c735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:08.893756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-4600e15b-fb65-4fc0-b178-93454208c735 None None] MFA Rules not processed for user `b9625ffd62304f9b943ea5325ea253f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:52:08.894953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-4600e15b-fb65-4fc0-b178-93454208c735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:08.958895 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1360/2749] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:52:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:52:08.970170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631e260-1547-4a42-a5b5-df77fdc9df0d admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:08.971987 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-4631e260-1547-4a42-a5b5-df77fdc9df0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:09.027470 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631e260-1547-4a42-a5b5-df77fdc9df0d admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:09.031173 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0b4af-5f78-456b-8e7b-1a2c3460ce07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:09.031763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0b4af-5f78-456b-8e7b-1a2c3460ce07 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:09.032064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0b4af-5f78-456b-8e7b-1a2c3460ce07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:09.032329 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0b4af-5f78-456b-8e7b-1a2c3460ce07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:09.041787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0b4af-5f78-456b-8e7b-1a2c3460ce07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:09.043243 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0b4af-5f78-456b-8e7b-1a2c3460ce07 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:09.052612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1390/2750] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:52:08 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:52:17.938348 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600e15b-fb65-4fc0-b178-93454208c735 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:18.007222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600e15b-fb65-4fc0-b178-93454208c735 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:18.012358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:18.012975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:18.013245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:18.013509 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] PATH_INFO: `/v3/users/b9625ffd62304f9b943ea5325ea253f4` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:18.014641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9625ffd62304f9b943ea5325ea253f4)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:18.016531 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:18.147134 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9625ffd62304f9b943ea5325ea253f4'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:18.147785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9625ffd62304f9b943ea5325ea253f4'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:18.189883 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9625ffd62304f9b943ea5325ea253f4'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:18.192092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:52:18.197768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9625ffd62304f9b943ea5325ea253f4'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:18.198421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9625ffd62304f9b943ea5325ea253f4'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:18.214898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1361/2751] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:52:17 2022] DELETE /identity/v3/users/b9625ffd62304f9b943ea5325ea253f4 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:52:18.219377 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2360e-a53f-4062-958d-222a18e4a6a3 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:18.219881 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2360e-a53f-4062-958d-222a18e4a6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:18.219984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2360e-a53f-4062-958d-222a18e4a6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:18.238855 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-89d2360e-a53f-4062-958d-222a18e4a6a3 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:52:18.240074 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-89d2360e-a53f-4062-958d-222a18e4a6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:18.300191 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1391/2752] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:52:18.310738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:18.312148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:18.369520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce4a9f-5878-4fe3-8514-3d380e96fcbf admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:18.373080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b266020-afb9-482f-b61b-aa342cba60d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:18.373672 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b266020-afb9-482f-b61b-aa342cba60d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:18.373987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b266020-afb9-482f-b61b-aa342cba60d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:18.374253 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b266020-afb9-482f-b61b-aa342cba60d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:18.383020 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b266020-afb9-482f-b61b-aa342cba60d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:18.384428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b266020-afb9-482f-b61b-aa342cba60d8 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:18.394243 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1362/2753] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:52:18.826324 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2360e-a53f-4062-958d-222a18e4a6a3 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:18.833945 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-89d2360e-a53f-4062-958d-222a18e4a6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:18.846223 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2360e-a53f-4062-958d-222a18e4a6a3 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:18.851333 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a203d7-9f29-4616-a842-50e8a3be7602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:18.852102 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a203d7-9f29-4616-a842-50e8a3be7602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:18.852384 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a203d7-9f29-4616-a842-50e8a3be7602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:18.852656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a203d7-9f29-4616-a842-50e8a3be7602 admin admin] PATH_INFO: `/v3/projects/994a2a4d9e704104a661c0392254223f` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:18.854522 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a203d7-9f29-4616-a842-50e8a3be7602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=994a2a4d9e704104a661c0392254223f)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:18.856361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a203d7-9f29-4616-a842-50e8a3be7602 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:18.969672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:52:18.970598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e5a203d7-9f29-4616-a842-50e8a3be7602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '994a2a4d9e704104a661c0392254223f', 'request_id': 'req-e5a203d7-9f29-4616-a842-50e8a3be7602'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:18.976193 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e5a203d7-9f29-4616-a842-50e8a3be7602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '994a2a4d9e704104a661c0392254223f', 'request_id': 'req-e5a203d7-9f29-4616-a842-50e8a3be7602'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:18.984163 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e5a203d7-9f29-4616-a842-50e8a3be7602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '994a2a4d9e704104a661c0392254223f', 'request_id': 'req-e5a203d7-9f29-4616-a842-50e8a3be7602'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:19.053177 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e5a203d7-9f29-4616-a842-50e8a3be7602 admin admin] The token cache is being invalidate because project 994a2a4d9e704104a661c0392254223f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:52:19.053489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e5a203d7-9f29-4616-a842-50e8a3be7602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:19.054324 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-e5a203d7-9f29-4616-a842-50e8a3be7602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:52:19.057898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1392/2754] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:52:18 2022] DELETE /identity/v3/projects/994a2a4d9e704104a661c0392254223f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:52:19.063122 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f018e-ca5f-45bd-b652-5bb576866e9d None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:19.064760 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f018e-ca5f-45bd-b652-5bb576866e9d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:19.065058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f018e-ca5f-45bd-b652-5bb576866e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:19.087573 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-098f018e-ca5f-45bd-b652-5bb576866e9d None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:52:19.088692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-098f018e-ca5f-45bd-b652-5bb576866e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:19.166729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1363/2755] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:52:19.176803 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0804674-c299-4598-84d5-ad84c0b784f0 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:19.177085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0804674-c299-4598-84d5-ad84c0b784f0 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:19.177358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0804674-c299-4598-84d5-ad84c0b784f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:19.201403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a0804674-c299-4598-84d5-ad84c0b784f0 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:52:19.202777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a0804674-c299-4598-84d5-ad84c0b784f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:19.213850 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1393/2756] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:52:19.218297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f018e-ca5f-45bd-b652-5bb576866e9d None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:19.230073 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f018e-ca5f-45bd-b652-5bb576866e9d None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:19.234964 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae696a4-3a3e-4d0c-b4ef-ceb40ae44a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:19.235564 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae696a4-3a3e-4d0c-b4ef-ceb40ae44a14 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:19.235838 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae696a4-3a3e-4d0c-b4ef-ceb40ae44a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:19.236105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae696a4-3a3e-4d0c-b4ef-ceb40ae44a14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:19.236749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae696a4-3a3e-4d0c-b4ef-ceb40ae44a14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:52:19.237045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae696a4-3a3e-4d0c-b4ef-ceb40ae44a14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:19.238688 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae696a4-3a3e-4d0c-b4ef-ceb40ae44a14 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:19.252864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1364/2757] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:52:19.260825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0804674-c299-4598-84d5-ad84c0b784f0 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:19.270676 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0804674-c299-4598-84d5-ad84c0b784f0 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:19.276365 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd71cf-7aca-4f4b-b5f2-cb2c1abd85ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:19.276960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd71cf-7aca-4f4b-b5f2-cb2c1abd85ec admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:19.277239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd71cf-7aca-4f4b-b5f2-cb2c1abd85ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:19.280319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd71cf-7aca-4f4b-b5f2-cb2c1abd85ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:19.280319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd71cf-7aca-4f4b-b5f2-cb2c1abd85ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:19.280319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbd71cf-7aca-4f4b-b5f2-cb2c1abd85ec admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:19.566360 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1394/2758] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:52:19 2022] POST /identity/v3/projects => generated 379 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:52:19.570918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae696a4-3a3e-4d0c-b4ef-ceb40ae44a14 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:19.581840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae696a4-3a3e-4d0c-b4ef-ceb40ae44a14 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:19.651365 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04dabb4-8bf1-44ee-b628-658119b26217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:19.651986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04dabb4-8bf1-44ee-b628-658119b26217 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:19.652260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04dabb4-8bf1-44ee-b628-658119b26217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:19.652527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04dabb4-8bf1-44ee-b628-658119b26217 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:19.653173 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04dabb4-8bf1-44ee-b628-658119b26217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:19.655351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04dabb4-8bf1-44ee-b628-658119b26217 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:19.666058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:52:19.759112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1365/2759] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:52:19 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:52:19.760459 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd71cf-7aca-4f4b-b5f2-cb2c1abd85ec admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:19.770715 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd71cf-7aca-4f4b-b5f2-cb2c1abd85ec admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:19.775441 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba46f6-564f-44ef-be23-9ed7a44e45a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:19.776358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba46f6-564f-44ef-be23-9ed7a44e45a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:19.776642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba46f6-564f-44ef-be23-9ed7a44e45a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:19.777008 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba46f6-564f-44ef-be23-9ed7a44e45a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:19.777572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba46f6-564f-44ef-be23-9ed7a44e45a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:19.779211 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba46f6-564f-44ef-be23-9ed7a44e45a4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:19.793324 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1395/2760] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:52:19.799078 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04dabb4-8bf1-44ee-b628-658119b26217 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:19.805073 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04dabb4-8bf1-44ee-b628-658119b26217 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:19.810294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135c0bb-53b3-401b-8ea2-956cebba409c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:19.810889 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135c0bb-53b3-401b-8ea2-956cebba409c admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:19.811163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135c0bb-53b3-401b-8ea2-956cebba409c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:19.811429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135c0bb-53b3-401b-8ea2-956cebba409c admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:19.812062 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135c0bb-53b3-401b-8ea2-956cebba409c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:19.813637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135c0bb-53b3-401b-8ea2-956cebba409c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:19.824629 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1366/2761] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:52:19.828669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba46f6-564f-44ef-be23-9ed7a44e45a4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:19.839225 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba46f6-564f-44ef-be23-9ed7a44e45a4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:19.843752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bee2ea-cc24-4269-adcc-29716f76bdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:19.844401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bee2ea-cc24-4269-adcc-29716f76bdfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:19.844676 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bee2ea-cc24-4269-adcc-29716f76bdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:19.844946 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bee2ea-cc24-4269-adcc-29716f76bdfc admin admin] PATH_INFO: `/v3/projects/22b431b5f9f344a6b72c3ee49d1c29ce/users/e6147d4c0d534cb2b580fda7fc2272ad/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:19.860215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bee2ea-cc24-4269-adcc-29716f76bdfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b431b5f9f344a6b72c3ee49d1c29ce, user_id=e6147d4c0d534cb2b580fda7fc2272ad, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:19.862893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bee2ea-cc24-4269-adcc-29716f76bdfc admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:19.939307 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1396/2762] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:52:19 2022] PUT /identity/v3/projects/22b431b5f9f344a6b72c3ee49d1c29ce/users/e6147d4c0d534cb2b580fda7fc2272ad/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:52:19.944351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac8da4-245b-4034-a16a-cbd93f531e39 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:19.944630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac8da4-245b-4034-a16a-cbd93f531e39 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:19.944898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac8da4-245b-4034-a16a-cbd93f531e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:19.964103 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-1fac8da4-245b-4034-a16a-cbd93f531e39 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:52:19.965382 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1fac8da4-245b-4034-a16a-cbd93f531e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:20.028773 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1367/2763] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:52:20.038351 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bee2ea-cc24-4269-adcc-29716f76bdfc admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:20.040305 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-92bee2ea-cc24-4269-adcc-29716f76bdfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:20.099421 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bee2ea-cc24-4269-adcc-29716f76bdfc admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:20.102966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4161d2-029a-4b86-be7d-7464df83c64c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:20.103572 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4161d2-029a-4b86-be7d-7464df83c64c service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:20.104483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4161d2-029a-4b86-be7d-7464df83c64c service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:20.104755 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4161d2-029a-4b86-be7d-7464df83c64c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:20.114167 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4161d2-029a-4b86-be7d-7464df83c64c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:20.115592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4161d2-029a-4b86-be7d-7464df83c64c service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:20.125406 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1397/2764] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:52:25.312749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d8d8b-8a98-4fbc-8b0f-683996bee544 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:25.313197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d8d8b-8a98-4fbc-8b0f-683996bee544 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:25.313617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d8d8b-8a98-4fbc-8b0f-683996bee544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:25.378323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-113d8d8b-8a98-4fbc-8b0f-683996bee544 None None] MFA Rules not processed for user `e6147d4c0d534cb2b580fda7fc2272ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:52:25.379526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-113d8d8b-8a98-4fbc-8b0f-683996bee544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:25.459057 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1368/2765] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:52:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:52:25.468910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4161d2-029a-4b86-be7d-7464df83c64c service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:25.483913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4161d2-029a-4b86-be7d-7464df83c64c service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:25.488579 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6338780-15d8-41f6-aed0-c1ae5c40134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:25.494981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6338780-15d8-41f6-aed0-c1ae5c40134b admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:25.494981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6338780-15d8-41f6-aed0-c1ae5c40134b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:25.494981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6338780-15d8-41f6-aed0-c1ae5c40134b admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:25.494981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6338780-15d8-41f6-aed0-c1ae5c40134b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:25.494981 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6338780-15d8-41f6-aed0-c1ae5c40134b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:25.575145 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1398/2766] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:52:25 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:52:25.580651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d8d8b-8a98-4fbc-8b0f-683996bee544 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:25.595227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d8d8b-8a98-4fbc-8b0f-683996bee544 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:25.656308 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb17de-ad22-45c5-916d-2f5c4647ba4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:25.656922 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb17de-ad22-45c5-916d-2f5c4647ba4c admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:25.657191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb17de-ad22-45c5-916d-2f5c4647ba4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:25.657453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eb17de-ad22-45c5-916d-2f5c4647ba4c admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:25.658120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb17de-ad22-45c5-916d-2f5c4647ba4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:25.660036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eb17de-ad22-45c5-916d-2f5c4647ba4c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:25.775545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1369/2767] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:52:25 2022] POST /identity/v3/users => generated 295 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:52:25.777668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6338780-15d8-41f6-aed0-c1ae5c40134b admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:25.794331 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6338780-15d8-41f6-aed0-c1ae5c40134b admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:25.800967 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbceda-c862-405b-a415-003f8c1da6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:25.801919 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbceda-c862-405b-a415-003f8c1da6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:25.802201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbceda-c862-405b-a415-003f8c1da6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:25.802475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbceda-c862-405b-a415-003f8c1da6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:25.803135 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbceda-c862-405b-a415-003f8c1da6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:25.804745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbceda-c862-405b-a415-003f8c1da6b9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:25.824861 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1399/2768] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:52:25.829101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb17de-ad22-45c5-916d-2f5c4647ba4c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:25.838660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:52:25.844725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eb17de-ad22-45c5-916d-2f5c4647ba4c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:25.849700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2859202-a52e-4511-8c65-fddec8e8eeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:25.850453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2859202-a52e-4511-8c65-fddec8e8eeca admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:25.850729 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2859202-a52e-4511-8c65-fddec8e8eeca admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:25.850999 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2859202-a52e-4511-8c65-fddec8e8eeca admin admin] PATH_INFO: `/v3/projects/c8ecac97ea8c4737a02c0a7c145b4b91/users/a122b9caa0414fc1bad0dbf5c7b86220/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:25.866949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2859202-a52e-4511-8c65-fddec8e8eeca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ecac97ea8c4737a02c0a7c145b4b91, user_id=a122b9caa0414fc1bad0dbf5c7b86220, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:25.869847 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2859202-a52e-4511-8c65-fddec8e8eeca admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:25.939520 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1370/2769] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:52:25 2022] PUT /identity/v3/projects/c8ecac97ea8c4737a02c0a7c145b4b91/users/a122b9caa0414fc1bad0dbf5c7b86220/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:52:25.943470 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbceda-c862-405b-a415-003f8c1da6b9 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:25.954577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbceda-c862-405b-a415-003f8c1da6b9 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:25.996558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:52:26.014022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf2bc6-b874-47b4-8280-6986bf0cd1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:26.014564 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf2bc6-b874-47b4-8280-6986bf0cd1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:26.014843 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf2bc6-b874-47b4-8280-6986bf0cd1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:26.015115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf2bc6-b874-47b4-8280-6986bf0cd1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:26.015778 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf2bc6-b874-47b4-8280-6986bf0cd1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:26.017379 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf2bc6-b874-47b4-8280-6986bf0cd1d9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:26.028287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1400/2770] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:52:26.032482 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2859202-a52e-4511-8c65-fddec8e8eeca admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:26.047632 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2859202-a52e-4511-8c65-fddec8e8eeca admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:26.054087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f653a7a-ca26-4433-970b-297f3d613ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:26.054770 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f653a7a-ca26-4433-970b-297f3d613ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:26.055041 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f653a7a-ca26-4433-970b-297f3d613ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:26.055309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f653a7a-ca26-4433-970b-297f3d613ce7 admin admin] PATH_INFO: `/v3/projects/c8ecac97ea8c4737a02c0a7c145b4b91/users/a122b9caa0414fc1bad0dbf5c7b86220/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:26.058036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f653a7a-ca26-4433-970b-297f3d613ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ecac97ea8c4737a02c0a7c145b4b91, user_id=a122b9caa0414fc1bad0dbf5c7b86220, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:26.062150 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f653a7a-ca26-4433-970b-297f3d613ce7 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:26.071651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1371/2771] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:52:26 2022] PUT /identity/v3/projects/c8ecac97ea8c4737a02c0a7c145b4b91/users/a122b9caa0414fc1bad0dbf5c7b86220/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:52:32.641134 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ca437-0a0e-4493-8ce6-6a4a4e8d176d None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:32.641607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ca437-0a0e-4493-8ce6-6a4a4e8d176d None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:32.641728 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ca437-0a0e-4493-8ce6-6a4a4e8d176d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:32.680972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-ab6ca437-0a0e-4493-8ce6-6a4a4e8d176d None None] MFA Rules not processed for user `a122b9caa0414fc1bad0dbf5c7b86220`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:52:32.683153 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-ab6ca437-0a0e-4493-8ce6-6a4a4e8d176d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:32.758045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1401/2772] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:52:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:52:32.773143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f653a7a-ca26-4433-970b-297f3d613ce7 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:32.775313 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5f653a7a-ca26-4433-970b-297f3d613ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:32.877631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f653a7a-ca26-4433-970b-297f3d613ce7 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:32.883285 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85967fda-fa25-489b-913d-64f5495ac2d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:32.883891 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85967fda-fa25-489b-913d-64f5495ac2d5 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:32.884168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85967fda-fa25-489b-913d-64f5495ac2d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:32.884458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85967fda-fa25-489b-913d-64f5495ac2d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:32.951704 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85967fda-fa25-489b-913d-64f5495ac2d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:32.953343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85967fda-fa25-489b-913d-64f5495ac2d5 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:32.962295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1372/2773] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:52:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:52:35.877704 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c5226-efd0-4e55-9e31-720927daa836 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:35.878255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c5226-efd0-4e55-9e31-720927daa836 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:35.878369 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c5226-efd0-4e55-9e31-720927daa836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:35.899226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-5e1c5226-efd0-4e55-9e31-720927daa836 None None] MFA Rules not processed for user `2dcce7be8c974a75a6ac82845865006f`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:52:35.901177 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-5e1c5226-efd0-4e55-9e31-720927daa836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:35.981483 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1402/2774] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:52:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:52:36.021350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85967fda-fa25-489b-913d-64f5495ac2d5 service nova] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:36.024013 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-85967fda-fa25-489b-913d-64f5495ac2d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:36.080120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:52:36.092051 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85967fda-fa25-489b-913d-64f5495ac2d5 service nova] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:36.095750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89edd2ed-a090-4358-8c8a-88cdbedbfb72 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30d96ec6235843c5b67d87a3d0ff0f8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:36.096353 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edd2ed-a090-4358-8c8a-88cdbedbfb72 service glance] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:36.096625 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edd2ed-a090-4358-8c8a-88cdbedbfb72 service glance] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:36.097023 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edd2ed-a090-4358-8c8a-88cdbedbfb72 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:36.098824 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-89edd2ed-a090-4358-8c8a-88cdbedbfb72 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:36.112708 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89edd2ed-a090-4358-8c8a-88cdbedbfb72 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:36.114199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89edd2ed-a090-4358-8c8a-88cdbedbfb72 service glance] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:36.126593 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1373/2775] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:52:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:52:36.842210 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c5226-efd0-4e55-9e31-720927daa836 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:52:36.901650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c5226-efd0-4e55-9e31-720927daa836 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:52:36.905208 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f4f80-bed4-4b73-a9a3-a78b56ad8d2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:52:36.905812 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f4f80-bed4-4b73-a9a3-a78b56ad8d2b service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:36.906109 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f4f80-bed4-4b73-a9a3-a78b56ad8d2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:36.906376 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f4f80-bed4-4b73-a9a3-a78b56ad8d2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:36.908319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e71f4f80-bed4-4b73-a9a3-a78b56ad8d2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:36.925368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f4f80-bed4-4b73-a9a3-a78b56ad8d2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:52:36.928050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f4f80-bed4-4b73-a9a3-a78b56ad8d2b service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:52:36.939545 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1403/2776] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:52:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:52:46.190290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=7 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:52:46.934681 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:52:51.400608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7007969-7234-400e-b08f-8bb4f7aa142c None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:52:51.401135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7007969-7234-400e-b08f-8bb4f7aa142c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:52:51.401249 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7007969-7234-400e-b08f-8bb4f7aa142c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:52:51.421270 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-d7007969-7234-400e-b08f-8bb4f7aa142c None None] MFA Rules not processed for user `e6147d4c0d534cb2b580fda7fc2272ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:52:51.422492 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d7007969-7234-400e-b08f-8bb4f7aa142c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:52:51.429344 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1374/2777] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:52:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:53:01.415361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:53:10.344728 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f4f80-bed4-4b73-a9a3-a78b56ad8d2b service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:10.346319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e71f4f80-bed4-4b73-a9a3-a78b56ad8d2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:10.356842 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f4f80-bed4-4b73-a9a3-a78b56ad8d2b service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:10.360386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7c9f1-6f95-437a-9915-899ee821910f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:10.361509 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7c9f1-6f95-437a-9915-899ee821910f service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:10.361775 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7c9f1-6f95-437a-9915-899ee821910f service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:10.362071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7c9f1-6f95-437a-9915-899ee821910f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:10.371350 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7c9f1-6f95-437a-9915-899ee821910f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:10.372956 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7c9f1-6f95-437a-9915-899ee821910f service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:10.383807 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1404/2778] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:53:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:53:10.927965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b16054-9c42-45ef-86d1-e4cf37bef0b7 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:10.928479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b16054-9c42-45ef-86d1-e4cf37bef0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:10.928596 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b16054-9c42-45ef-86d1-e4cf37bef0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:10.983992 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-70b16054-9c42-45ef-86d1-e4cf37bef0b7 None None] MFA Rules not processed for user `ba7a3df6fb6a4eb3a077a21b40794754`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:53:10.985246 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-70b16054-9c42-45ef-86d1-e4cf37bef0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:11.055389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1375/2779] 10.222.0.49 () {58 vars in 967 bytes} [Fri Sep 16 05:53:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:53:11.223484 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7c9f1-6f95-437a-9915-899ee821910f service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:11.224924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c1c7c9f1-6f95-437a-9915-899ee821910f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:11.323443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7c9f1-6f95-437a-9915-899ee821910f service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:11.328592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6bbf7-d617-4e28-84c6-361c25da6671 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b260fed0d99a4d3fa8f9c2ad79223219', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:11.329192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6bbf7-d617-4e28-84c6-361c25da6671 service swift] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:11.329467 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6bbf7-d617-4e28-84c6-361c25da6671 service swift] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:11.330402 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6bbf7-d617-4e28-84c6-361c25da6671 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:11.347489 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6bbf7-d617-4e28-84c6-361c25da6671 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:11.349122 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6bbf7-d617-4e28-84c6-361c25da6671 service swift] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:11.357615 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1405/2780] 10.222.0.49 () {60 vars in 1384 bytes} [Fri Sep 16 05:53:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 16 05:53:12.563666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b16054-9c42-45ef-86d1-e4cf37bef0b7 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:12.577104 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b16054-9c42-45ef-86d1-e4cf37bef0b7 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:12.583840 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8dddd-9836-41fe-aac1-9c59fb4e0c33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:12.584649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8dddd-9836-41fe-aac1-9c59fb4e0c33 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:12.586050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8dddd-9836-41fe-aac1-9c59fb4e0c33 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:12.586351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8dddd-9836-41fe-aac1-9c59fb4e0c33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:12.588490 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-23c8dddd-9836-41fe-aac1-9c59fb4e0c33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:12.604615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8dddd-9836-41fe-aac1-9c59fb4e0c33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:12.606661 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8dddd-9836-41fe-aac1-9c59fb4e0c33 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:12.626844 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1376/2781] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:53:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:53:12.723496 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6bbf7-d617-4e28-84c6-361c25da6671 service swift] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:12.733138 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6bbf7-d617-4e28-84c6-361c25da6671 service swift] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:12.743596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88559adf-d631-4380-8c9b-94ae3ee938c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:12.743596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88559adf-d631-4380-8c9b-94ae3ee938c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:12.743596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88559adf-d631-4380-8c9b-94ae3ee938c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:12.743596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88559adf-d631-4380-8c9b-94ae3ee938c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:12.743998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-88559adf-d631-4380-8c9b-94ae3ee938c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:12.751436 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88559adf-d631-4380-8c9b-94ae3ee938c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:12.753801 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88559adf-d631-4380-8c9b-94ae3ee938c0 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:12.762939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1406/2782] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:53:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:53:19.855132 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8dddd-9836-41fe-aac1-9c59fb4e0c33 service nova] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:19.856567 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-23c8dddd-9836-41fe-aac1-9c59fb4e0c33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:19.934017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8dddd-9836-41fe-aac1-9c59fb4e0c33 service nova] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:19.938541 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:19.939254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:19.939607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:19.940087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] PATH_INFO: `/v3/users/7092b595cffa4ba48d45982165ac27b0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:19.941454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7092b595cffa4ba48d45982165ac27b0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:19.943389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:20.135429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7092b595cffa4ba48d45982165ac27b0'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:20.136080 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7092b595cffa4ba48d45982165ac27b0'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:20.212494 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7092b595cffa4ba48d45982165ac27b0'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:20.218060 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:53:20.220131 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7092b595cffa4ba48d45982165ac27b0'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:20.221307 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7092b595cffa4ba48d45982165ac27b0'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:20.239292 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1377/2783] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:53:19 2022] DELETE /identity/v3/users/7092b595cffa4ba48d45982165ac27b0 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:53:20.453488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88559adf-d631-4380-8c9b-94ae3ee938c0 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:20.468954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88559adf-d631-4380-8c9b-94ae3ee938c0 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:20.532531 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:20.534155 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:20.534630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:20.535039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] PATH_INFO: `/v3/projects/d0f3aae0f9814d189d002373cc873f42` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:20.536421 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f3aae0f9814d189d002373cc873f42)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:20.538344 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:20.617934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:53:20.620743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f3aae0f9814d189d002373cc873f42', 'request_id': 'req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:20.626176 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f3aae0f9814d189d002373cc873f42', 'request_id': 'req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:20.633041 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f3aae0f9814d189d002373cc873f42', 'request_id': 'req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:20.708696 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] The token cache is being invalidate because project d0f3aae0f9814d189d002373cc873f42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:53:20.709198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:20.710233 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:20.711720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1407/2784] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:53:20 2022] DELETE /identity/v3/projects/d0f3aae0f9814d189d002373cc873f42 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:53:20.715812 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:20.717233 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:20.731256 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e91fe-551f-48fd-b539-7a27cfc468cd admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:20.787743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:20.788360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:20.788629 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:20.788895 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] PATH_INFO: `/v3/users/d6c7ae6eb7be41a6a3d1d0996d7fdb37` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:20.790029 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6c7ae6eb7be41a6a3d1d0996d7fdb37)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:20.792198 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:20.976574 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6c7ae6eb7be41a6a3d1d0996d7fdb37'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:20.977230 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6c7ae6eb7be41a6a3d1d0996d7fdb37'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:21.064444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6c7ae6eb7be41a6a3d1d0996d7fdb37'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:21.072282 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6c7ae6eb7be41a6a3d1d0996d7fdb37'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:21.073782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6c7ae6eb7be41a6a3d1d0996d7fdb37'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:21.090736 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1378/2785] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:53:20 2022] DELETE /identity/v3/users/d6c7ae6eb7be41a6a3d1d0996d7fdb37 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:53:21.460170 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:21.461668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:21.476659 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6f2a2-87d3-4bab-b6eb-4c4ecda5d82d admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:21.535588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:21.536392 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:21.536669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:21.536943 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] PATH_INFO: `/v3/projects/4105a8ce820c4deeb0a3d46d49d5a481` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:21.538109 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4105a8ce820c4deeb0a3d46d49d5a481)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:21.540007 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:21.648338 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4105a8ce820c4deeb0a3d46d49d5a481', 'request_id': 'req-b1ed543d-22c5-45f3-a056-d13fc733e9cc'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:21.653763 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4105a8ce820c4deeb0a3d46d49d5a481', 'request_id': 'req-b1ed543d-22c5-45f3-a056-d13fc733e9cc'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:21.660820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4105a8ce820c4deeb0a3d46d49d5a481', 'request_id': 'req-b1ed543d-22c5-45f3-a056-d13fc733e9cc'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:21.738116 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] The token cache is being invalidate because project 4105a8ce820c4deeb0a3d46d49d5a481 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:53:21.738319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:21.738938 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:21.740427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1408/2786] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:53:21 2022] DELETE /identity/v3/projects/4105a8ce820c4deeb0a3d46d49d5a481 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:53:31.087184 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=61,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:53:31.734292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:DELETE=11,keystone:SELECT=58,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:53:40.441056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:40.444119 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:40.530756 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c420032-d4db-41a8-ab9d-a7e82f4a3744 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:40.615354 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcd280-6978-4bc4-bd23-1c82575ec9bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:40.616138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcd280-6978-4bc4-bd23-1c82575ec9bc service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:40.616731 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcd280-6978-4bc4-bd23-1c82575ec9bc service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:40.617099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcd280-6978-4bc4-bd23-1c82575ec9bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:40.619175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-95dcd280-6978-4bc4-bd23-1c82575ec9bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:40.633530 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcd280-6978-4bc4-bd23-1c82575ec9bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:40.635417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dcd280-6978-4bc4-bd23-1c82575ec9bc service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:40.648198 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1379/2787] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:53:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:53:44.750900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:44.750900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:44.806592 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed543d-22c5-45f3-a056-d13fc733e9cc admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:44.810267 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d47f2a-a5fa-4817-b9a7-c07a647b1a88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:44.810903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d47f2a-a5fa-4817-b9a7-c07a647b1a88 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:44.811180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d47f2a-a5fa-4817-b9a7-c07a647b1a88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:44.811448 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d47f2a-a5fa-4817-b9a7-c07a647b1a88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:44.813142 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-46d47f2a-a5fa-4817-b9a7-c07a647b1a88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:44.872297 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d47f2a-a5fa-4817-b9a7-c07a647b1a88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:44.873701 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d47f2a-a5fa-4817-b9a7-c07a647b1a88 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:44.883477 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1409/2788] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:53:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:53:47.448421 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcd280-6978-4bc4-bd23-1c82575ec9bc service nova] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:47.449785 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-95dcd280-6978-4bc4-bd23-1c82575ec9bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:47.504217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcd280-6978-4bc4-bd23-1c82575ec9bc service nova] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:47.508574 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ee28a-b61a-43dc-8754-c663a9179f3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:47.509213 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ee28a-b61a-43dc-8754-c663a9179f3a service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:47.509485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ee28a-b61a-43dc-8754-c663a9179f3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:47.509751 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ee28a-b61a-43dc-8754-c663a9179f3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:47.511474 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-c20ee28a-b61a-43dc-8754-c663a9179f3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:47.550584 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:53:47.576293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ee28a-b61a-43dc-8754-c663a9179f3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:47.577698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ee28a-b61a-43dc-8754-c663a9179f3a service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:47.588077 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1380/2789] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:53:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:53:51.057980 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d47f2a-a5fa-4817-b9a7-c07a647b1a88 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:51.059928 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-46d47f2a-a5fa-4817-b9a7-c07a647b1a88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:51.074800 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d47f2a-a5fa-4817-b9a7-c07a647b1a88 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:51.079474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686c058-b5af-4297-a230-0f75ccdaa892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:51.080109 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686c058-b5af-4297-a230-0f75ccdaa892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:51.080380 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686c058-b5af-4297-a230-0f75ccdaa892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:51.080672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686c058-b5af-4297-a230-0f75ccdaa892 admin admin] PATH_INFO: `/v3/users/2dcce7be8c974a75a6ac82845865006f` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:51.081784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686c058-b5af-4297-a230-0f75ccdaa892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dcce7be8c974a75a6ac82845865006f)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:51.084371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686c058-b5af-4297-a230-0f75ccdaa892 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:51.282481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-0686c058-b5af-4297-a230-0f75ccdaa892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dcce7be8c974a75a6ac82845865006f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:51.283115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-0686c058-b5af-4297-a230-0f75ccdaa892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dcce7be8c974a75a6ac82845865006f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:51.361192 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-0686c058-b5af-4297-a230-0f75ccdaa892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dcce7be8c974a75a6ac82845865006f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:51.361996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-0686c058-b5af-4297-a230-0f75ccdaa892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dcce7be8c974a75a6ac82845865006f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:51.370276 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-0686c058-b5af-4297-a230-0f75ccdaa892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dcce7be8c974a75a6ac82845865006f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:51.388218 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1410/2790] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:53:51 2022] DELETE /identity/v3/users/2dcce7be8c974a75a6ac82845865006f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:53:51.638070 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ee28a-b61a-43dc-8754-c663a9179f3a service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:51.656666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ee28a-b61a-43dc-8754-c663a9179f3a service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:51.716438 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:51.717059 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:51.717330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:51.717591 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c admin admin] PATH_INFO: `/v3/projects/86f59bdb0c3a46bf86fe611b95335b19` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:51.718770 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c admin admin] RBAC: Authorizing `identity:delete_project(project_id=86f59bdb0c3a46bf86fe611b95335b19)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:51.724752 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:51.829550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86f59bdb0c3a46bf86fe611b95335b19', 'request_id': 'req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:51.835281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86f59bdb0c3a46bf86fe611b95335b19', 'request_id': 'req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:51.841456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86f59bdb0c3a46bf86fe611b95335b19', 'request_id': 'req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:51.902960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c admin admin] The token cache is being invalidate because project 86f59bdb0c3a46bf86fe611b95335b19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:53:51.903245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:51.903861 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-ac1aa0f3-249e-4e89-9d4d-e62929b9021c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:53:51.905340 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1381/2791] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:53:51 2022] DELETE /identity/v3/projects/86f59bdb0c3a46bf86fe611b95335b19 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:53:51.910265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef558a3-17c8-406b-8444-dee4ee6fd67f None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:51.910540 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef558a3-17c8-406b-8444-dee4ee6fd67f None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:51.910809 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef558a3-17c8-406b-8444-dee4ee6fd67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:51.932707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-fef558a3-17c8-406b-8444-dee4ee6fd67f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:53:51.933930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-fef558a3-17c8-406b-8444-dee4ee6fd67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:51.953908 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:53:51.992160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1411/2792] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:53:52.000091 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65c91b-9708-4150-b33f-c7cb739bc75a None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:52.000375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65c91b-9708-4150-b33f-c7cb739bc75a None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:52.000644 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65c91b-9708-4150-b33f-c7cb739bc75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:52.023533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-5f65c91b-9708-4150-b33f-c7cb739bc75a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:53:52.024666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5f65c91b-9708-4150-b33f-c7cb739bc75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:52.032800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1382/2793] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:53:52.037845 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef558a3-17c8-406b-8444-dee4ee6fd67f None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:52.047751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef558a3-17c8-406b-8444-dee4ee6fd67f None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:52.052353 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aceaec-d871-47bc-aafc-2c9c7c2c1ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:52.053194 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aceaec-d871-47bc-aafc-2c9c7c2c1ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:52.053466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aceaec-d871-47bc-aafc-2c9c7c2c1ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:52.053731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aceaec-d871-47bc-aafc-2c9c7c2c1ccf admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:52.054398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aceaec-d871-47bc-aafc-2c9c7c2c1ccf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:53:52.054695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aceaec-d871-47bc-aafc-2c9c7c2c1ccf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:52.056265 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aceaec-d871-47bc-aafc-2c9c7c2c1ccf admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:52.067525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1412/2794] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:53:52.071740 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65c91b-9708-4150-b33f-c7cb739bc75a None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:52.080780 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65c91b-9708-4150-b33f-c7cb739bc75a None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:52.086195 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be8f32-e838-4f61-a157-5923beacf54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:52.086795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be8f32-e838-4f61-a157-5923beacf54a admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:52.087067 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be8f32-e838-4f61-a157-5923beacf54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:52.087330 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be8f32-e838-4f61-a157-5923beacf54a admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:52.087961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be8f32-e838-4f61-a157-5923beacf54a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:52.089842 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be8f32-e838-4f61-a157-5923beacf54a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:52.188907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1383/2795] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:53:52 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:53:52.190157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aceaec-d871-47bc-aafc-2c9c7c2c1ccf admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:52.200401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aceaec-d871-47bc-aafc-2c9c7c2c1ccf admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:52.255234 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab5f92-997c-4861-9e2c-10e7d7a15fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:52.256090 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab5f92-997c-4861-9e2c-10e7d7a15fca admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:52.256398 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab5f92-997c-4861-9e2c-10e7d7a15fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:52.256668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab5f92-997c-4861-9e2c-10e7d7a15fca admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:52.257285 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab5f92-997c-4861-9e2c-10e7d7a15fca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:52.259220 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab5f92-997c-4861-9e2c-10e7d7a15fca admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:52.544684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1413/2796] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:53:52 2022] POST /identity/v3/users => generated 283 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:53:52.548782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be8f32-e838-4f61-a157-5923beacf54a admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:52.559006 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be8f32-e838-4f61-a157-5923beacf54a admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:52.563539 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65379b8-96bd-44c1-938a-393e9ba1fdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:52.564135 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65379b8-96bd-44c1-938a-393e9ba1fdb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:52.564407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65379b8-96bd-44c1-938a-393e9ba1fdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:52.564671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65379b8-96bd-44c1-938a-393e9ba1fdb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:52.565304 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65379b8-96bd-44c1-938a-393e9ba1fdb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:52.566974 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65379b8-96bd-44c1-938a-393e9ba1fdb6 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:52.577316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1384/2797] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:53:52.586480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab5f92-997c-4861-9e2c-10e7d7a15fca admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:52.597070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab5f92-997c-4861-9e2c-10e7d7a15fca admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:52.603226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbbd87-5c48-4669-85f0-57e2d3169065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:52.603855 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbbd87-5c48-4669-85f0-57e2d3169065 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:52.604131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbbd87-5c48-4669-85f0-57e2d3169065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:52.604400 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbbd87-5c48-4669-85f0-57e2d3169065 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:52.605035 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbbd87-5c48-4669-85f0-57e2d3169065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:52.606651 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbbd87-5c48-4669-85f0-57e2d3169065 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:52.617972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1414/2798] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:53:52.622111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65379b8-96bd-44c1-938a-393e9ba1fdb6 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:52.632673 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65379b8-96bd-44c1-938a-393e9ba1fdb6 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:52.637545 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab747f3-7dc0-4f7b-92df-a08de8214a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:52.641000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab747f3-7dc0-4f7b-92df-a08de8214a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:52.641000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab747f3-7dc0-4f7b-92df-a08de8214a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:52.641000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab747f3-7dc0-4f7b-92df-a08de8214a7e admin admin] PATH_INFO: `/v3/projects/8b6ab55134554d83882cedd242a46fcf/users/336992ee0d634f9c81f47ab01469d172/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:52.660279 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab747f3-7dc0-4f7b-92df-a08de8214a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b6ab55134554d83882cedd242a46fcf, user_id=336992ee0d634f9c81f47ab01469d172, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:52.664115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab747f3-7dc0-4f7b-92df-a08de8214a7e admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:52.734101 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:53:52.738748 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1385/2799] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:53:52 2022] PUT /identity/v3/projects/8b6ab55134554d83882cedd242a46fcf/users/336992ee0d634f9c81f47ab01469d172/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:53:52.750453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11144fe-ba1c-4795-bade-4da789819a76 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:52.756235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11144fe-ba1c-4795-bade-4da789819a76 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:52.756235 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11144fe-ba1c-4795-bade-4da789819a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:52.778732 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-a11144fe-ba1c-4795-bade-4da789819a76 None None] MFA Rules not processed for user `336992ee0d634f9c81f47ab01469d172`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:53:52.779916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-a11144fe-ba1c-4795-bade-4da789819a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:52.843623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1415/2800] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:53:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:53:52.854534 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab747f3-7dc0-4f7b-92df-a08de8214a7e admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:53:52.856271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-aab747f3-7dc0-4f7b-92df-a08de8214a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:53:52.914485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab747f3-7dc0-4f7b-92df-a08de8214a7e admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:53:52.919264 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48098602-9af4-476a-a4fa-708b46b23c61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:53:52.919875 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48098602-9af4-476a-a4fa-708b46b23c61 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:53:52.920148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48098602-9af4-476a-a4fa-708b46b23c61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:53:52.920414 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48098602-9af4-476a-a4fa-708b46b23c61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:53:52.929692 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48098602-9af4-476a-a4fa-708b46b23c61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:53:52.931128 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48098602-9af4-476a-a4fa-708b46b23c61 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:53:52.939821 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1386/2801] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:53:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:54:02.613069 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11144fe-ba1c-4795-bade-4da789819a76 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:02.642515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:54:02.679685 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11144fe-ba1c-4795-bade-4da789819a76 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:02.684223 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:02.684855 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:02.685130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:02.685397 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] PATH_INFO: `/v3/users/336992ee0d634f9c81f47ab01469d172` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:02.686656 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] RBAC: Authorizing `identity:delete_user(user_id=336992ee0d634f9c81f47ab01469d172)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:02.688904 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:02.843276 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336992ee0d634f9c81f47ab01469d172'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:02.843840 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336992ee0d634f9c81f47ab01469d172'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:02.907969 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336992ee0d634f9c81f47ab01469d172'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:02.908604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336992ee0d634f9c81f47ab01469d172'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:02.915906 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336992ee0d634f9c81f47ab01469d172'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:02.934318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1416/2802] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:54:02 2022] DELETE /identity/v3/users/336992ee0d634f9c81f47ab01469d172 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:54:02.935871 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=26 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:54:02.938743 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb7159-24f5-43fe-847a-f1772908dbc2 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:02.939022 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb7159-24f5-43fe-847a-f1772908dbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:02.939287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb7159-24f5-43fe-847a-f1772908dbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:02.958711 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-7acb7159-24f5-43fe-847a-f1772908dbc2 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:54:02.959895 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7acb7159-24f5-43fe-847a-f1772908dbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:03.017444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1387/2803] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:54:03.027878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:03.029273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:03.086146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b64da1-e451-4ead-9709-5fd8360ab98e admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:03.090580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd2f04-8b2d-4ae7-81f6-ddc46612d935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:03.091181 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd2f04-8b2d-4ae7-81f6-ddc46612d935 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:03.091461 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd2f04-8b2d-4ae7-81f6-ddc46612d935 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:03.091730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd2f04-8b2d-4ae7-81f6-ddc46612d935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:03.100886 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd2f04-8b2d-4ae7-81f6-ddc46612d935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:03.102284 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd2f04-8b2d-4ae7-81f6-ddc46612d935 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:03.111957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1417/2804] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:54:03.476560 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb7159-24f5-43fe-847a-f1772908dbc2 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:03.480937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-7acb7159-24f5-43fe-847a-f1772908dbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:03.490153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb7159-24f5-43fe-847a-f1772908dbc2 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:03.495262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:03.495900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:03.496179 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:03.496442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea admin admin] PATH_INFO: `/v3/projects/8b6ab55134554d83882cedd242a46fcf` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:03.497551 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b6ab55134554d83882cedd242a46fcf)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:03.499485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:03.606422 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b6ab55134554d83882cedd242a46fcf', 'request_id': 'req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:03.611848 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b6ab55134554d83882cedd242a46fcf', 'request_id': 'req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:03.617907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b6ab55134554d83882cedd242a46fcf', 'request_id': 'req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:03.680182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea admin admin] The token cache is being invalidate because project 8b6ab55134554d83882cedd242a46fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:54:03.680468 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:03.681085 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-259f1d8b-2078-4eb0-b90e-b4aa5c84ccea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:03.683165 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1388/2805] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:54:03 2022] DELETE /identity/v3/projects/8b6ab55134554d83882cedd242a46fcf => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:54:03.688346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97551ba8-545d-492e-88a8-a9fa15f0357a None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:03.688812 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97551ba8-545d-492e-88a8-a9fa15f0357a None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:03.688924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97551ba8-545d-492e-88a8-a9fa15f0357a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:03.710691 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-97551ba8-545d-492e-88a8-a9fa15f0357a None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:54:03.711922 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-97551ba8-545d-492e-88a8-a9fa15f0357a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:03.753126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:54:03.771198 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1418/2806] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:54:03.779470 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6f91c-1cca-46ac-86c1-2019a54854b4 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:03.779750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6f91c-1cca-46ac-86c1-2019a54854b4 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:03.780017 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6f91c-1cca-46ac-86c1-2019a54854b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:03.800960 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-78e6f91c-1cca-46ac-86c1-2019a54854b4 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:54:03.802187 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-78e6f91c-1cca-46ac-86c1-2019a54854b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:03.810968 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1389/2807] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:54:03.815356 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97551ba8-545d-492e-88a8-a9fa15f0357a None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:03.825367 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97551ba8-545d-492e-88a8-a9fa15f0357a None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:03.829983 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d072bcd-5c50-4e62-b468-927abf4d2f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:03.830696 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d072bcd-5c50-4e62-b468-927abf4d2f3b admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:03.830974 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d072bcd-5c50-4e62-b468-927abf4d2f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:03.831240 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d072bcd-5c50-4e62-b468-927abf4d2f3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:03.831878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d072bcd-5c50-4e62-b468-927abf4d2f3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:54:03.832174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d072bcd-5c50-4e62-b468-927abf4d2f3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:03.833743 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d072bcd-5c50-4e62-b468-927abf4d2f3b admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:03.845104 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1419/2808] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:54:03.849319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6f91c-1cca-46ac-86c1-2019a54854b4 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:03.858958 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6f91c-1cca-46ac-86c1-2019a54854b4 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:03.863414 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bc299-e680-476c-bd85-46cddbd93f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:03.863997 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bc299-e680-476c-bd85-46cddbd93f3d admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:03.864269 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bc299-e680-476c-bd85-46cddbd93f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:03.864532 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bc299-e680-476c-bd85-46cddbd93f3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:03.865160 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33bc299-e680-476c-bd85-46cddbd93f3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:03.867076 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33bc299-e680-476c-bd85-46cddbd93f3d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:03.939417 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1390/2809] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:54:03 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:54:03.943731 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d072bcd-5c50-4e62-b468-927abf4d2f3b admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:03.954694 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d072bcd-5c50-4e62-b468-927abf4d2f3b admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:04.012346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c0bf1-9bbb-4628-9ead-41caa466c715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:04.012949 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c0bf1-9bbb-4628-9ead-41caa466c715 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:04.013224 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c0bf1-9bbb-4628-9ead-41caa466c715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:04.013491 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c0bf1-9bbb-4628-9ead-41caa466c715 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:04.014159 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c0bf1-9bbb-4628-9ead-41caa466c715 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:04.016060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c0bf1-9bbb-4628-9ead-41caa466c715 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:04.097966 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1420/2810] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:54:03 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:54:04.102236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bc299-e680-476c-bd85-46cddbd93f3d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:04.113339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bc299-e680-476c-bd85-46cddbd93f3d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:04.118197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b0a7c-e9c0-4947-8493-8d1f389e2413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:04.118796 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b0a7c-e9c0-4947-8493-8d1f389e2413 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:04.119279 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b0a7c-e9c0-4947-8493-8d1f389e2413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:04.119606 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b0a7c-e9c0-4947-8493-8d1f389e2413 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:04.120271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b0a7c-e9c0-4947-8493-8d1f389e2413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:04.121850 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b0a7c-e9c0-4947-8493-8d1f389e2413 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:04.131705 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1391/2811] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:54:04.136115 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c0bf1-9bbb-4628-9ead-41caa466c715 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:04.146353 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c0bf1-9bbb-4628-9ead-41caa466c715 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:04.151557 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6334f-b3c7-423d-b46c-5c5d2930a5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:04.152150 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6334f-b3c7-423d-b46c-5c5d2930a5be admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:04.152425 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6334f-b3c7-423d-b46c-5c5d2930a5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:04.152697 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6334f-b3c7-423d-b46c-5c5d2930a5be admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:04.153337 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6334f-b3c7-423d-b46c-5c5d2930a5be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:04.154946 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6334f-b3c7-423d-b46c-5c5d2930a5be admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:04.165430 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1421/2812] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:54:04.170012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b0a7c-e9c0-4947-8493-8d1f389e2413 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:04.179795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b0a7c-e9c0-4947-8493-8d1f389e2413 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:04.184671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff21758-348d-44d5-9e23-91a16ac3a3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:04.185303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21758-348d-44d5-9e23-91a16ac3a3b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:04.185575 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21758-348d-44d5-9e23-91a16ac3a3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:04.185864 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21758-348d-44d5-9e23-91a16ac3a3b9 admin admin] PATH_INFO: `/v3/projects/d2ff1caa075748a39a74230bbe4454e2/users/372f6f50191e485ab14c3f017c4965df/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:04.199929 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff21758-348d-44d5-9e23-91a16ac3a3b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ff1caa075748a39a74230bbe4454e2, user_id=372f6f50191e485ab14c3f017c4965df, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:04.202589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff21758-348d-44d5-9e23-91a16ac3a3b9 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:04.287032 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1392/2813] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:54:04 2022] PUT /identity/v3/projects/d2ff1caa075748a39a74230bbe4454e2/users/372f6f50191e485ab14c3f017c4965df/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:54:04.299131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c36ad6-a11e-41e0-81ad-0a3125a0115e None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:04.299414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c36ad6-a11e-41e0-81ad-0a3125a0115e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:04.299712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c36ad6-a11e-41e0-81ad-0a3125a0115e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:04.330403 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-b7c36ad6-a11e-41e0-81ad-0a3125a0115e None None] MFA Rules not processed for user `372f6f50191e485ab14c3f017c4965df`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:54:04.331648 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-b7c36ad6-a11e-41e0-81ad-0a3125a0115e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:04.396872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1422/2814] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:54:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:54:04.401933 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff21758-348d-44d5-9e23-91a16ac3a3b9 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:04.423032 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:54:04.463741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff21758-348d-44d5-9e23-91a16ac3a3b9 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:04.469114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c270cb-63fe-4b8c-b43b-0b9ea4fc9a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:04.469716 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c270cb-63fe-4b8c-b43b-0b9ea4fc9a4a admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:04.470037 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c270cb-63fe-4b8c-b43b-0b9ea4fc9a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:04.470309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c270cb-63fe-4b8c-b43b-0b9ea4fc9a4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:04.470942 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c270cb-63fe-4b8c-b43b-0b9ea4fc9a4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:04.472866 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c270cb-63fe-4b8c-b43b-0b9ea4fc9a4a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:04.556519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1393/2815] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:54:04 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:54:04.560644 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c36ad6-a11e-41e0-81ad-0a3125a0115e None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:04.573096 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c36ad6-a11e-41e0-81ad-0a3125a0115e None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:04.611568 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:54:04.634182 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec908a1f-687c-41f4-9c78-a7ccb512efe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:04.635295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec908a1f-687c-41f4-9c78-a7ccb512efe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:04.635591 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec908a1f-687c-41f4-9c78-a7ccb512efe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:04.635931 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec908a1f-687c-41f4-9c78-a7ccb512efe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:04.636563 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec908a1f-687c-41f4-9c78-a7ccb512efe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:04.638508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec908a1f-687c-41f4-9c78-a7ccb512efe6 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:04.753295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1423/2816] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:54:04 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:54:04.757407 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c270cb-63fe-4b8c-b43b-0b9ea4fc9a4a admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:04.768763 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c270cb-63fe-4b8c-b43b-0b9ea4fc9a4a admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:04.773288 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c370a8-6f7c-4fc3-8d01-826495bb855a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:04.773907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c370a8-6f7c-4fc3-8d01-826495bb855a admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:04.774180 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c370a8-6f7c-4fc3-8d01-826495bb855a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:04.774447 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c370a8-6f7c-4fc3-8d01-826495bb855a admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:04.775082 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c370a8-6f7c-4fc3-8d01-826495bb855a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:04.777327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c370a8-6f7c-4fc3-8d01-826495bb855a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:04.787644 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1394/2817] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:54:04.792022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec908a1f-687c-41f4-9c78-a7ccb512efe6 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:04.802066 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec908a1f-687c-41f4-9c78-a7ccb512efe6 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:04.807290 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558a294-b57b-4588-85c8-8fb1c7432684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:04.808200 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558a294-b57b-4588-85c8-8fb1c7432684 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:04.808598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558a294-b57b-4588-85c8-8fb1c7432684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:04.809003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3558a294-b57b-4588-85c8-8fb1c7432684 admin admin] PATH_INFO: `/v3/projects/a47db6a63a0245ecb2f2426a1cd13783/users/2ca23f9a903b40af85b2ea942caaae96/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:04.824371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558a294-b57b-4588-85c8-8fb1c7432684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a47db6a63a0245ecb2f2426a1cd13783, user_id=2ca23f9a903b40af85b2ea942caaae96, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:04.827049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3558a294-b57b-4588-85c8-8fb1c7432684 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:04.938559 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1424/2818] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:54:04 2022] PUT /identity/v3/projects/a47db6a63a0245ecb2f2426a1cd13783/users/2ca23f9a903b40af85b2ea942caaae96/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:54:04.942507 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c370a8-6f7c-4fc3-8d01-826495bb855a admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:04.953186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c370a8-6f7c-4fc3-8d01-826495bb855a admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:05.011351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaaf01a-12b4-4e3c-9e95-7a3b45574408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:05.011871 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaf01a-12b4-4e3c-9e95-7a3b45574408 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:05.012147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaf01a-12b4-4e3c-9e95-7a3b45574408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:05.012414 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaf01a-12b4-4e3c-9e95-7a3b45574408 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:05.013044 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaaf01a-12b4-4e3c-9e95-7a3b45574408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:05.014663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaaf01a-12b4-4e3c-9e95-7a3b45574408 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:05.024042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1395/2819] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:54:05.028337 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558a294-b57b-4588-85c8-8fb1c7432684 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:05.038806 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3558a294-b57b-4588-85c8-8fb1c7432684 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:05.044201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2dd24-92bd-4847-842b-57183b96d09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:05.044841 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2dd24-92bd-4847-842b-57183b96d09f admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:05.045118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2dd24-92bd-4847-842b-57183b96d09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:05.045387 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2dd24-92bd-4847-842b-57183b96d09f admin admin] PATH_INFO: `/v3/projects/a47db6a63a0245ecb2f2426a1cd13783/users/2ca23f9a903b40af85b2ea942caaae96/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:05.047353 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2dd24-92bd-4847-842b-57183b96d09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a47db6a63a0245ecb2f2426a1cd13783, user_id=2ca23f9a903b40af85b2ea942caaae96, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:05.050006 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2dd24-92bd-4847-842b-57183b96d09f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:05.060750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1425/2820] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:54:05 2022] PUT /identity/v3/projects/a47db6a63a0245ecb2f2426a1cd13783/users/2ca23f9a903b40af85b2ea942caaae96/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:54:05.070715 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9d231-d31a-4fb5-8094-f88efd116f47 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:05.071000 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9d231-d31a-4fb5-8094-f88efd116f47 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:05.071578 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9d231-d31a-4fb5-8094-f88efd116f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:05.099201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-6dd9d231-d31a-4fb5-8094-f88efd116f47 None None] MFA Rules not processed for user `2ca23f9a903b40af85b2ea942caaae96`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:54:05.100414 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-6dd9d231-d31a-4fb5-8094-f88efd116f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:05.167373 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1396/2821] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:54:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:54:05.178481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2dd24-92bd-4847-842b-57183b96d09f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:05.183085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8fe2dd24-92bd-4847-842b-57183b96d09f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:05.237469 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2dd24-92bd-4847-842b-57183b96d09f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:05.241000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5f86d-0fee-4d19-a530-ea7ec0f600d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:05.241584 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5f86d-0fee-4d19-a530-ea7ec0f600d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:05.241886 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5f86d-0fee-4d19-a530-ea7ec0f600d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:05.242154 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5f86d-0fee-4d19-a530-ea7ec0f600d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:05.251664 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5f86d-0fee-4d19-a530-ea7ec0f600d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:05.253068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5f86d-0fee-4d19-a530-ea7ec0f600d3 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:05.262400 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1426/2822] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:54:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:54:05.437254 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9d231-d31a-4fb5-8094-f88efd116f47 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:05.446900 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9d231-d31a-4fb5-8094-f88efd116f47 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:05.450396 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd07a8-1649-4b7a-baf4-4d3246ebce48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:05.450988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd07a8-1649-4b7a-baf4-4d3246ebce48 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:05.451263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd07a8-1649-4b7a-baf4-4d3246ebce48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:05.451533 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd07a8-1649-4b7a-baf4-4d3246ebce48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:05.463589 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:54:05.510122 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd07a8-1649-4b7a-baf4-4d3246ebce48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:05.511577 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd07a8-1649-4b7a-baf4-4d3246ebce48 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:05.520700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1397/2823] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:54:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:54:15.258158 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=52,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:54:15.521641 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=21 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:54:51.585577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5f86d-0fee-4d19-a530-ea7ec0f600d3 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:51.647880 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5f86d-0fee-4d19-a530-ea7ec0f600d3 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:51.652500 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:51.653123 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:51.653401 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:51.653669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] PATH_INFO: `/v3/users/372f6f50191e485ab14c3f017c4965df` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:51.654973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=372f6f50191e485ab14c3f017c4965df)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:51.656991 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:51.852346 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '372f6f50191e485ab14c3f017c4965df'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:51.852924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '372f6f50191e485ab14c3f017c4965df'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:51.924645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '372f6f50191e485ab14c3f017c4965df'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:51.925255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '372f6f50191e485ab14c3f017c4965df'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:51.932860 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '372f6f50191e485ab14c3f017c4965df'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:51.950724 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1427/2824] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:54:51 2022] DELETE /identity/v3/users/372f6f50191e485ab14c3f017c4965df => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:54:51.955332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af020f-7779-4453-a750-647a7a55da0f None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:51.955873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af020f-7779-4453-a750-647a7a55da0f None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:51.955873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af020f-7779-4453-a750-647a7a55da0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:51.974508 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-57af020f-7779-4453-a750-647a7a55da0f None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:54:51.975673 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-57af020f-7779-4453-a750-647a7a55da0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:52.033008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1398/2825] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:54:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:54:52.043061 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:52.044967 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:52.100405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025ea10-90c3-47fc-b625-946efbbbebcc admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:52.104120 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c2e6e-4d3d-40ba-9fce-9b1c93e2b3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:52.104658 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c2e6e-4d3d-40ba-9fce-9b1c93e2b3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:52.104947 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c2e6e-4d3d-40ba-9fce-9b1c93e2b3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:52.105204 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c2e6e-4d3d-40ba-9fce-9b1c93e2b3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:52.114632 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c2e6e-4d3d-40ba-9fce-9b1c93e2b3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:52.116049 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c2e6e-4d3d-40ba-9fce-9b1c93e2b3b6 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:52.125932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1428/2826] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:54:52.508659 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af020f-7779-4453-a750-647a7a55da0f None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:52.511687 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-57af020f-7779-4453-a750-647a7a55da0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:52.521549 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af020f-7779-4453-a750-647a7a55da0f None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:52.527098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:52.527727 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:52.528067 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:52.528295 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] PATH_INFO: `/v3/projects/d2ff1caa075748a39a74230bbe4454e2` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:52.530389 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ff1caa075748a39a74230bbe4454e2)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:52.532333 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:52.664432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ff1caa075748a39a74230bbe4454e2', 'request_id': 'req-55e379ab-4d03-411a-ba93-175fb6c836d6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:52.671189 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ff1caa075748a39a74230bbe4454e2', 'request_id': 'req-55e379ab-4d03-411a-ba93-175fb6c836d6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:52.677540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ff1caa075748a39a74230bbe4454e2', 'request_id': 'req-55e379ab-4d03-411a-ba93-175fb6c836d6'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:52.769910 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] The token cache is being invalidate because project d2ff1caa075748a39a74230bbe4454e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:54:52.770235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:52.770858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:52.772429 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1399/2827] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:54:52 2022] DELETE /identity/v3/projects/d2ff1caa075748a39a74230bbe4454e2 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:54:52.776498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c2e6e-4d3d-40ba-9fce-9b1c93e2b3b6 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:52.786709 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c2e6e-4d3d-40ba-9fce-9b1c93e2b3b6 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:52.805791 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:54:52.845393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a8270-435f-447f-98c0-e0cbf13060a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:52.846054 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a8270-435f-447f-98c0-e0cbf13060a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:52.846327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a8270-435f-447f-98c0-e0cbf13060a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:52.846593 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a8270-435f-447f-98c0-e0cbf13060a6 admin admin] PATH_INFO: `/v3/users/2ca23f9a903b40af85b2ea942caaae96` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:52.847680 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a8270-435f-447f-98c0-e0cbf13060a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca23f9a903b40af85b2ea942caaae96)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:52.849673 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a8270-435f-447f-98c0-e0cbf13060a6 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:53.047542 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-037a8270-435f-447f-98c0-e0cbf13060a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca23f9a903b40af85b2ea942caaae96'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:53.048189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-037a8270-435f-447f-98c0-e0cbf13060a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca23f9a903b40af85b2ea942caaae96'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:53.112898 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-037a8270-435f-447f-98c0-e0cbf13060a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca23f9a903b40af85b2ea942caaae96'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:53.113566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-037a8270-435f-447f-98c0-e0cbf13060a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca23f9a903b40af85b2ea942caaae96'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:53.120862 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-037a8270-435f-447f-98c0-e0cbf13060a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca23f9a903b40af85b2ea942caaae96'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:53.138112 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1429/2828] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:54:52 2022] DELETE /identity/v3/users/2ca23f9a903b40af85b2ea942caaae96 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:54:53.567887 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:53.569413 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:53.583489 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e379ab-4d03-411a-ba93-175fb6c836d6 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:53.661317 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09a63c-dc42-4b08-9966-7d0526ccc170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:53.661989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09a63c-dc42-4b08-9966-7d0526ccc170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:53.662259 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09a63c-dc42-4b08-9966-7d0526ccc170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:53.662523 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09a63c-dc42-4b08-9966-7d0526ccc170 admin admin] PATH_INFO: `/v3/projects/a47db6a63a0245ecb2f2426a1cd13783` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:53.663630 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09a63c-dc42-4b08-9966-7d0526ccc170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a47db6a63a0245ecb2f2426a1cd13783)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:53.665568 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09a63c-dc42-4b08-9966-7d0526ccc170 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:53.923182 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:54:53.939755 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fb09a63c-dc42-4b08-9966-7d0526ccc170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a47db6a63a0245ecb2f2426a1cd13783', 'request_id': 'req-fb09a63c-dc42-4b08-9966-7d0526ccc170'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:53.946741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fb09a63c-dc42-4b08-9966-7d0526ccc170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a47db6a63a0245ecb2f2426a1cd13783', 'request_id': 'req-fb09a63c-dc42-4b08-9966-7d0526ccc170'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:53.954432 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fb09a63c-dc42-4b08-9966-7d0526ccc170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a47db6a63a0245ecb2f2426a1cd13783', 'request_id': 'req-fb09a63c-dc42-4b08-9966-7d0526ccc170'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:54.032377 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fb09a63c-dc42-4b08-9966-7d0526ccc170 admin admin] The token cache is being invalidate because project a47db6a63a0245ecb2f2426a1cd13783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:54:54.032655 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fb09a63c-dc42-4b08-9966-7d0526ccc170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:54.033266 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fb09a63c-dc42-4b08-9966-7d0526ccc170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:54:54.034758 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1400/2829] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:54:53 2022] DELETE /identity/v3/projects/a47db6a63a0245ecb2f2426a1cd13783 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:54:54.039598 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3beae-353f-4020-9317-52e1290fbdb2 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:54.040087 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3beae-353f-4020-9317-52e1290fbdb2 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:54.040199 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3beae-353f-4020-9317-52e1290fbdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:54.061474 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-aff3beae-353f-4020-9317-52e1290fbdb2 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:54:54.062712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-aff3beae-353f-4020-9317-52e1290fbdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:54.120893 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1430/2830] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:54:54.181424 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cbcba-9ebb-44cc-a91d-9c38f4f71f1e None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:54.181656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cbcba-9ebb-44cc-a91d-9c38f4f71f1e None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:54.181970 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cbcba-9ebb-44cc-a91d-9c38f4f71f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:54.201277 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-661cbcba-9ebb-44cc-a91d-9c38f4f71f1e None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:54:54.202485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-661cbcba-9ebb-44cc-a91d-9c38f4f71f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:54.210143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1401/2831] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:54:54.214364 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3beae-353f-4020-9317-52e1290fbdb2 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:54.225036 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3beae-353f-4020-9317-52e1290fbdb2 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:54.229777 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5a2c5-c34d-4bec-ab84-e15dd487de54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:54.230411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5a2c5-c34d-4bec-ab84-e15dd487de54 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:54.230692 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5a2c5-c34d-4bec-ab84-e15dd487de54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:54.230959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5a2c5-c34d-4bec-ab84-e15dd487de54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:54.231605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5a2c5-c34d-4bec-ab84-e15dd487de54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:54:54.232586 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5a2c5-c34d-4bec-ab84-e15dd487de54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:54.234269 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5a2c5-c34d-4bec-ab84-e15dd487de54 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:54.245748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1431/2832] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:54:54.250110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cbcba-9ebb-44cc-a91d-9c38f4f71f1e None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:54.259117 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cbcba-9ebb-44cc-a91d-9c38f4f71f1e None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:54.263897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae181592-675b-44a9-9d32-944c89fa8af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:54.264489 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae181592-675b-44a9-9d32-944c89fa8af4 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:54.264764 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae181592-675b-44a9-9d32-944c89fa8af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:54.265033 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae181592-675b-44a9-9d32-944c89fa8af4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:54.265652 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae181592-675b-44a9-9d32-944c89fa8af4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:54.267557 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae181592-675b-44a9-9d32-944c89fa8af4 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:54.356026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1402/2833] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:54:54 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:54:54.361026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5a2c5-c34d-4bec-ab84-e15dd487de54 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:54.371323 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5a2c5-c34d-4bec-ab84-e15dd487de54 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:54.396605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:54:54.427473 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca014b7-0105-4e99-a9b5-8fe80d8adc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:54.428393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca014b7-0105-4e99-a9b5-8fe80d8adc25 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:54.428870 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca014b7-0105-4e99-a9b5-8fe80d8adc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:54.429548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca014b7-0105-4e99-a9b5-8fe80d8adc25 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:54.430255 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca014b7-0105-4e99-a9b5-8fe80d8adc25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:54.432241 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca014b7-0105-4e99-a9b5-8fe80d8adc25 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:54.524226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1432/2834] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:54:54 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:54:54.528464 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae181592-675b-44a9-9d32-944c89fa8af4 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:54.539460 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae181592-675b-44a9-9d32-944c89fa8af4 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:54.544358 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef12576-83c0-4737-8a9e-9738916dafb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:54.544994 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef12576-83c0-4737-8a9e-9738916dafb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:54.545279 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef12576-83c0-4737-8a9e-9738916dafb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:54.545547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef12576-83c0-4737-8a9e-9738916dafb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:54.546216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef12576-83c0-4737-8a9e-9738916dafb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:54.547793 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef12576-83c0-4737-8a9e-9738916dafb3 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:54.557478 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1403/2835] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:54:54.561661 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca014b7-0105-4e99-a9b5-8fe80d8adc25 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:54.571510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca014b7-0105-4e99-a9b5-8fe80d8adc25 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:54.576772 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbc675-58ad-4b80-a26a-bc52183be5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:54.577441 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbc675-58ad-4b80-a26a-bc52183be5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:54.577689 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbc675-58ad-4b80-a26a-bc52183be5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:54.577993 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbc675-58ad-4b80-a26a-bc52183be5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:54.578639 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbc675-58ad-4b80-a26a-bc52183be5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:54.580223 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbc675-58ad-4b80-a26a-bc52183be5c8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:54.591000 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1433/2836] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 16 05:54:54.595890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef12576-83c0-4737-8a9e-9738916dafb3 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:54.606099 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef12576-83c0-4737-8a9e-9738916dafb3 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:54.610710 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eaeb2d-af6b-4ca0-8dcc-2ea3cd17c2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:54.611348 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaeb2d-af6b-4ca0-8dcc-2ea3cd17c2b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:54.611622 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaeb2d-af6b-4ca0-8dcc-2ea3cd17c2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:54.611890 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eaeb2d-af6b-4ca0-8dcc-2ea3cd17c2b5 admin admin] PATH_INFO: `/v3/projects/199e67a4241047e3a9454034c00ac1bc/users/42c5f5d7171e41d7957352e64e66a991/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:54.626418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eaeb2d-af6b-4ca0-8dcc-2ea3cd17c2b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=199e67a4241047e3a9454034c00ac1bc, user_id=42c5f5d7171e41d7957352e64e66a991, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:54.629050 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eaeb2d-af6b-4ca0-8dcc-2ea3cd17c2b5 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:54.701039 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1404/2837] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:54:54 2022] PUT /identity/v3/projects/199e67a4241047e3a9454034c00ac1bc/users/42c5f5d7171e41d7957352e64e66a991/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:54:54.711274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12140d26-4949-4467-a8b0-3161c1e9f31c None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:54.711555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12140d26-4949-4467-a8b0-3161c1e9f31c None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:54.711825 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12140d26-4949-4467-a8b0-3161c1e9f31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:54.740838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-12140d26-4949-4467-a8b0-3161c1e9f31c None None] MFA Rules not processed for user `42c5f5d7171e41d7957352e64e66a991`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:54:54.742051 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-12140d26-4949-4467-a8b0-3161c1e9f31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:54.808110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1434/2838] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:54:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:54:54.813461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eaeb2d-af6b-4ca0-8dcc-2ea3cd17c2b5 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:54.875896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eaeb2d-af6b-4ca0-8dcc-2ea3cd17c2b5 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:54.880516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432bfdc-307f-4ce1-adad-373e60fc7d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:54.881107 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432bfdc-307f-4ce1-adad-373e60fc7d43 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:54.881383 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432bfdc-307f-4ce1-adad-373e60fc7d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:54.881653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0432bfdc-307f-4ce1-adad-373e60fc7d43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:54.882321 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0432bfdc-307f-4ce1-adad-373e60fc7d43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:54.884205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0432bfdc-307f-4ce1-adad-373e60fc7d43 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:54.985852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1405/2839] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:54:54 2022] POST /identity/v3/projects => generated 367 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:54:54.989852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12140d26-4949-4467-a8b0-3161c1e9f31c None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:55.003156 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12140d26-4949-4467-a8b0-3161c1e9f31c None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:55.058876 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f6b67-db51-43b6-bbd4-76e8fc067013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:55.059525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f6b67-db51-43b6-bbd4-76e8fc067013 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:55.059737 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f6b67-db51-43b6-bbd4-76e8fc067013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:55.060005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f6b67-db51-43b6-bbd4-76e8fc067013 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:55.061228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f6b67-db51-43b6-bbd4-76e8fc067013 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:55.063556 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f6b67-db51-43b6-bbd4-76e8fc067013 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:55.177959 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1435/2840] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:54:54 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:54:55.182010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432bfdc-307f-4ce1-adad-373e60fc7d43 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:55.192795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0432bfdc-307f-4ce1-adad-373e60fc7d43 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:55.197374 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d933a7-8285-4ee4-a4ac-968dfe6a13e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:55.198012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d933a7-8285-4ee4-a4ac-968dfe6a13e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:55.198290 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d933a7-8285-4ee4-a4ac-968dfe6a13e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:55.198559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d933a7-8285-4ee4-a4ac-968dfe6a13e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:55.199193 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d933a7-8285-4ee4-a4ac-968dfe6a13e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:55.200768 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d933a7-8285-4ee4-a4ac-968dfe6a13e5 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:55.211827 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1406/2841] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:54:55.215853 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f6b67-db51-43b6-bbd4-76e8fc067013 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:55.225871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f6b67-db51-43b6-bbd4-76e8fc067013 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:55.230548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f0e9b-a2bd-441c-97e1-e54b66299723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:55.231189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f0e9b-a2bd-441c-97e1-e54b66299723 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:55.231490 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f0e9b-a2bd-441c-97e1-e54b66299723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:55.231757 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f0e9b-a2bd-441c-97e1-e54b66299723 admin admin] PATH_INFO: `/v3/projects/6b72f7baae8b483d9a451dc6525d2355/users/078e8a6a895e428cb2c73bbd5fcb9bf9/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:55.246830 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f0e9b-a2bd-441c-97e1-e54b66299723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b72f7baae8b483d9a451dc6525d2355, user_id=078e8a6a895e428cb2c73bbd5fcb9bf9, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:55.250039 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f0e9b-a2bd-441c-97e1-e54b66299723 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:55.320741 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1436/2842] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:54:55 2022] PUT /identity/v3/projects/6b72f7baae8b483d9a451dc6525d2355/users/078e8a6a895e428cb2c73bbd5fcb9bf9/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:54:55.324856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d933a7-8285-4ee4-a4ac-968dfe6a13e5 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:55.334810 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d933a7-8285-4ee4-a4ac-968dfe6a13e5 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:55.356479 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:54:55.396797 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596992f-35b6-4678-b4b9-cec52326d1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:55.399163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596992f-35b6-4678-b4b9-cec52326d1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:55.399163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596992f-35b6-4678-b4b9-cec52326d1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:55.399163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596992f-35b6-4678-b4b9-cec52326d1e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:55.399163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596992f-35b6-4678-b4b9-cec52326d1e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:55.400097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596992f-35b6-4678-b4b9-cec52326d1e2 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:55.410228 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1407/2843] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:54:55.414422 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f0e9b-a2bd-441c-97e1-e54b66299723 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:55.420079 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:54:55.425492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f0e9b-a2bd-441c-97e1-e54b66299723 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:55.430316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2877d-b50a-475e-91b0-359ecb958ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:55.430979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2877d-b50a-475e-91b0-359ecb958ad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:55.431249 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2877d-b50a-475e-91b0-359ecb958ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:55.431516 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2877d-b50a-475e-91b0-359ecb958ad8 admin admin] PATH_INFO: `/v3/projects/6b72f7baae8b483d9a451dc6525d2355/users/078e8a6a895e428cb2c73bbd5fcb9bf9/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:55.433518 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2877d-b50a-475e-91b0-359ecb958ad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b72f7baae8b483d9a451dc6525d2355, user_id=078e8a6a895e428cb2c73bbd5fcb9bf9, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:55.436457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2877d-b50a-475e-91b0-359ecb958ad8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:55.448161 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1437/2844] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:54:55 2022] PUT /identity/v3/projects/6b72f7baae8b483d9a451dc6525d2355/users/078e8a6a895e428cb2c73bbd5fcb9bf9/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:54:55.460016 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346baf40-6ba8-430a-9342-995b4febe1a9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:55.460303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346baf40-6ba8-430a-9342-995b4febe1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:55.460580 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346baf40-6ba8-430a-9342-995b4febe1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:55.490468 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-346baf40-6ba8-430a-9342-995b4febe1a9 None None] MFA Rules not processed for user `078e8a6a895e428cb2c73bbd5fcb9bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:54:55.491698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-346baf40-6ba8-430a-9342-995b4febe1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:55.562519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1408/2845] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:54:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:54:55.575642 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2877d-b50a-475e-91b0-359ecb958ad8 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:55.578228 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-c8d2877d-b50a-475e-91b0-359ecb958ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:55.637580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2877d-b50a-475e-91b0-359ecb958ad8 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:55.641872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eceed93-b70d-4bf6-95d9-fdc14ee68858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:55.642475 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceed93-b70d-4bf6-95d9-fdc14ee68858 service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:55.642752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceed93-b70d-4bf6-95d9-fdc14ee68858 service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:55.643091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eceed93-b70d-4bf6-95d9-fdc14ee68858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:55.652404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eceed93-b70d-4bf6-95d9-fdc14ee68858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:55.653851 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eceed93-b70d-4bf6-95d9-fdc14ee68858 service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:55.664725 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1438/2846] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:54:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:54:55.823444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346baf40-6ba8-430a-9342-995b4febe1a9 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:54:55.823843 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-346baf40-6ba8-430a-9342-995b4febe1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:54:55.835965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346baf40-6ba8-430a-9342-995b4febe1a9 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:54:55.839558 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc212b7-437c-4255-b86c-9cd26584b29f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:54:55.840158 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc212b7-437c-4255-b86c-9cd26584b29f service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:54:55.840433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc212b7-437c-4255-b86c-9cd26584b29f service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:54:55.840703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc212b7-437c-4255-b86c-9cd26584b29f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:54:55.898394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc212b7-437c-4255-b86c-9cd26584b29f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:54:55.899819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc212b7-437c-4255-b86c-9cd26584b29f service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:54:55.909790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1409/2847] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:54:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:55:00.724309 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eceed93-b70d-4bf6-95d9-fdc14ee68858 service cinder] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:00.790437 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eceed93-b70d-4bf6-95d9-fdc14ee68858 service cinder] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:00.795262 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:00.795888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:00.796162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:00.796429 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] PATH_INFO: `/v3/users/42c5f5d7171e41d7957352e64e66a991` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:00.797611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42c5f5d7171e41d7957352e64e66a991)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:00.799550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:00.968900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42c5f5d7171e41d7957352e64e66a991'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:00.969543 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42c5f5d7171e41d7957352e64e66a991'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:01.034371 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42c5f5d7171e41d7957352e64e66a991'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:01.034980 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42c5f5d7171e41d7957352e64e66a991'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:01.042776 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42c5f5d7171e41d7957352e64e66a991'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:01.060195 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1439/2848] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:55:00 2022] DELETE /identity/v3/users/42c5f5d7171e41d7957352e64e66a991 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:01.065315 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52596d9d-7fe7-4a00-97e6-e564ab7c3b3c None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:01.067102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52596d9d-7fe7-4a00-97e6-e564ab7c3b3c None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:01.067102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52596d9d-7fe7-4a00-97e6-e564ab7c3b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:01.084950 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-52596d9d-7fe7-4a00-97e6-e564ab7c3b3c None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:01.086168 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-52596d9d-7fe7-4a00-97e6-e564ab7c3b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:01.124287 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=100 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:01.143550 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1410/2849] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:01.153130 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:01.158772 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:01.168650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:01.211588 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27fc1c3-6891-440a-9605-1fdfdaf2bcf8 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:01.215549 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab2c26-f5a3-442a-a0e1-76cda1fb586b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:01.216145 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab2c26-f5a3-442a-a0e1-76cda1fb586b service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:01.216414 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab2c26-f5a3-442a-a0e1-76cda1fb586b service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:01.216675 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab2c26-f5a3-442a-a0e1-76cda1fb586b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:01.227428 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab2c26-f5a3-442a-a0e1-76cda1fb586b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:01.227428 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab2c26-f5a3-442a-a0e1-76cda1fb586b service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:01.237239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1440/2850] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:55:01.657775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52596d9d-7fe7-4a00-97e6-e564ab7c3b3c None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:01.659322 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-52596d9d-7fe7-4a00-97e6-e564ab7c3b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:01.673394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52596d9d-7fe7-4a00-97e6-e564ab7c3b3c None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:01.678858 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:01.679504 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:01.679779 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:01.680699 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] PATH_INFO: `/v3/projects/199e67a4241047e3a9454034c00ac1bc` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:01.681228 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=199e67a4241047e3a9454034c00ac1bc)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:01.687647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:01.780932 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '199e67a4241047e3a9454034c00ac1bc', 'request_id': 'req-16ec82df-1f83-48e2-beed-9e429566ec74'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:01.786881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '199e67a4241047e3a9454034c00ac1bc', 'request_id': 'req-16ec82df-1f83-48e2-beed-9e429566ec74'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:01.793196 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '199e67a4241047e3a9454034c00ac1bc', 'request_id': 'req-16ec82df-1f83-48e2-beed-9e429566ec74'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:01.891300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] The token cache is being invalidate because project 199e67a4241047e3a9454034c00ac1bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:55:01.891834 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:01.892460 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:01.893955 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1411/2851] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:55:01 2022] DELETE /identity/v3/projects/199e67a4241047e3a9454034c00ac1bc => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:01.897890 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab2c26-f5a3-442a-a0e1-76cda1fb586b service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:01.908476 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab2c26-f5a3-442a-a0e1-76cda1fb586b service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:01.967454 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000451e3-4d07-4fd4-adc3-bf72220526d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:01.968068 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000451e3-4d07-4fd4-adc3-bf72220526d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:01.968340 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000451e3-4d07-4fd4-adc3-bf72220526d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:01.969292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000451e3-4d07-4fd4-adc3-bf72220526d4 admin admin] PATH_INFO: `/v3/users/078e8a6a895e428cb2c73bbd5fcb9bf9` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:01.970427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000451e3-4d07-4fd4-adc3-bf72220526d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=078e8a6a895e428cb2c73bbd5fcb9bf9)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:01.972316 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000451e3-4d07-4fd4-adc3-bf72220526d4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:02.157630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-000451e3-4d07-4fd4-adc3-bf72220526d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '078e8a6a895e428cb2c73bbd5fcb9bf9'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:02.158296 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-000451e3-4d07-4fd4-adc3-bf72220526d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '078e8a6a895e428cb2c73bbd5fcb9bf9'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:02.255645 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-000451e3-4d07-4fd4-adc3-bf72220526d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '078e8a6a895e428cb2c73bbd5fcb9bf9'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:02.256253 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-000451e3-4d07-4fd4-adc3-bf72220526d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '078e8a6a895e428cb2c73bbd5fcb9bf9'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:02.263764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-000451e3-4d07-4fd4-adc3-bf72220526d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '078e8a6a895e428cb2c73bbd5fcb9bf9'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:02.282157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1441/2852] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:55:01 2022] DELETE /identity/v3/users/078e8a6a895e428cb2c73bbd5fcb9bf9 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:02.692097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:02.693531 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:02.709965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec82df-1f83-48e2-beed-9e429566ec74 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:02.769715 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946b8ed-19ba-4ee6-90e5-849559e5442f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:02.770528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946b8ed-19ba-4ee6-90e5-849559e5442f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:02.771869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946b8ed-19ba-4ee6-90e5-849559e5442f admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:02.772156 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946b8ed-19ba-4ee6-90e5-849559e5442f admin admin] PATH_INFO: `/v3/projects/6b72f7baae8b483d9a451dc6525d2355` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:02.773685 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4946b8ed-19ba-4ee6-90e5-849559e5442f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b72f7baae8b483d9a451dc6525d2355)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:02.777437 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4946b8ed-19ba-4ee6-90e5-849559e5442f admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:02.879251 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4946b8ed-19ba-4ee6-90e5-849559e5442f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b72f7baae8b483d9a451dc6525d2355', 'request_id': 'req-4946b8ed-19ba-4ee6-90e5-849559e5442f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:02.884823 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4946b8ed-19ba-4ee6-90e5-849559e5442f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b72f7baae8b483d9a451dc6525d2355', 'request_id': 'req-4946b8ed-19ba-4ee6-90e5-849559e5442f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:02.891587 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4946b8ed-19ba-4ee6-90e5-849559e5442f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b72f7baae8b483d9a451dc6525d2355', 'request_id': 'req-4946b8ed-19ba-4ee6-90e5-849559e5442f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:02.982795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:02.987635 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4946b8ed-19ba-4ee6-90e5-849559e5442f admin admin] The token cache is being invalidate because project 6b72f7baae8b483d9a451dc6525d2355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:55:02.987919 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4946b8ed-19ba-4ee6-90e5-849559e5442f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:02.988536 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-4946b8ed-19ba-4ee6-90e5-849559e5442f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:02.990031 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1412/2853] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:55:02 2022] DELETE /identity/v3/projects/6b72f7baae8b483d9a451dc6525d2355 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:03.000146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852b00f-a680-4b3b-941c-f3101a621cbd None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:03.000146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852b00f-a680-4b3b-941c-f3101a621cbd None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:03.000146 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852b00f-a680-4b3b-941c-f3101a621cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:03.015157 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-f852b00f-a680-4b3b-941c-f3101a621cbd None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:03.016871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f852b00f-a680-4b3b-941c-f3101a621cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:03.045576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:03.074942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1442/2854] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:03.082995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3540b7-8280-4d98-bf61-8f587659e7a9 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:03.083291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3540b7-8280-4d98-bf61-8f587659e7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:03.083561 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3540b7-8280-4d98-bf61-8f587659e7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:03.103719 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-df3540b7-8280-4d98-bf61-8f587659e7a9 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:03.104897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-df3540b7-8280-4d98-bf61-8f587659e7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:03.115877 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1413/2855] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:03.117988 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852b00f-a680-4b3b-941c-f3101a621cbd None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:03.127821 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852b00f-a680-4b3b-941c-f3101a621cbd None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:03.132361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28c6a0-2882-4653-8e9f-ab652912d037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:03.132891 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28c6a0-2882-4653-8e9f-ab652912d037 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:03.133162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28c6a0-2882-4653-8e9f-ab652912d037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:03.133423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28c6a0-2882-4653-8e9f-ab652912d037 admin admin] PATH_INFO: `/v3/domains` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:03.134091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28c6a0-2882-4653-8e9f-ab652912d037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81557) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:55:03.134383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28c6a0-2882-4653-8e9f-ab652912d037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:03.135948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28c6a0-2882-4653-8e9f-ab652912d037 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:03.147970 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1443/2856] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:55:03.152107 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3540b7-8280-4d98-bf61-8f587659e7a9 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:03.161796 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3540b7-8280-4d98-bf61-8f587659e7a9 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:03.166546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f41a90-2b13-431a-98f5-c1aba828b5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:03.167394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41a90-2b13-431a-98f5-c1aba828b5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:03.167663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41a90-2b13-431a-98f5-c1aba828b5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:03.167925 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41a90-2b13-431a-98f5-c1aba828b5d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:03.168546 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f41a90-2b13-431a-98f5-c1aba828b5d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:03.170738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f41a90-2b13-431a-98f5-c1aba828b5d6 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:03.235988 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1414/2857] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:55:03 2022] POST /identity/v3/projects => generated 387 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:55:03.240283 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28c6a0-2882-4653-8e9f-ab652912d037 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:03.250180 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28c6a0-2882-4653-8e9f-ab652912d037 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:03.305481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b7ce4-543e-43c5-8de9-47fcca8028cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:03.306105 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b7ce4-543e-43c5-8de9-47fcca8028cf admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:03.306383 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b7ce4-543e-43c5-8de9-47fcca8028cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:03.306652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b7ce4-543e-43c5-8de9-47fcca8028cf admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:03.307273 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b7ce4-543e-43c5-8de9-47fcca8028cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:03.309165 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b7ce4-543e-43c5-8de9-47fcca8028cf admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:03.389251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1444/2858] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:55:03 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:55:03.394205 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f41a90-2b13-431a-98f5-c1aba828b5d6 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:03.404224 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f41a90-2b13-431a-98f5-c1aba828b5d6 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:03.409514 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a859fdb-19f0-42a3-b89d-05719c427b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:03.410138 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a859fdb-19f0-42a3-b89d-05719c427b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:03.410409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a859fdb-19f0-42a3-b89d-05719c427b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:03.410671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a859fdb-19f0-42a3-b89d-05719c427b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:03.411297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a859fdb-19f0-42a3-b89d-05719c427b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:03.412865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a859fdb-19f0-42a3-b89d-05719c427b7d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:03.422851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1415/2859] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:55:03.426796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b7ce4-543e-43c5-8de9-47fcca8028cf admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:03.436748 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b7ce4-543e-43c5-8de9-47fcca8028cf admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:03.441612 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afcde0-1f94-4e66-9e7e-ae642b5938c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:03.442226 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afcde0-1f94-4e66-9e7e-ae642b5938c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:03.442496 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afcde0-1f94-4e66-9e7e-ae642b5938c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:03.442761 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afcde0-1f94-4e66-9e7e-ae642b5938c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:03.443424 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afcde0-1f94-4e66-9e7e-ae642b5938c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:03.445026 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afcde0-1f94-4e66-9e7e-ae642b5938c3 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:03.456091 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1445/2860] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:55:03.460084 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a859fdb-19f0-42a3-b89d-05719c427b7d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:03.470649 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a859fdb-19f0-42a3-b89d-05719c427b7d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:03.475153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4267575-b285-4f33-9cb7-196ddf367521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:03.475786 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4267575-b285-4f33-9cb7-196ddf367521 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:03.476053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4267575-b285-4f33-9cb7-196ddf367521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:03.476371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4267575-b285-4f33-9cb7-196ddf367521 admin admin] PATH_INFO: `/v3/projects/f1458b9e14044c5cb4546079ed3b0bf4/users/532932a6351d48f0a2669203e7c9317f/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:03.491260 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4267575-b285-4f33-9cb7-196ddf367521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1458b9e14044c5cb4546079ed3b0bf4, user_id=532932a6351d48f0a2669203e7c9317f, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:03.494012 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4267575-b285-4f33-9cb7-196ddf367521 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:03.582120 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1416/2861] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:55:03 2022] PUT /identity/v3/projects/f1458b9e14044c5cb4546079ed3b0bf4/users/532932a6351d48f0a2669203e7c9317f/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:03.592385 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fba79-1553-47fa-9faa-bff8b1222202 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:03.592665 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fba79-1553-47fa-9faa-bff8b1222202 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:03.592932 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fba79-1553-47fa-9faa-bff8b1222202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:03.620638 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-380fba79-1553-47fa-9faa-bff8b1222202 None None] MFA Rules not processed for user `532932a6351d48f0a2669203e7c9317f`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:03.621843 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-380fba79-1553-47fa-9faa-bff8b1222202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:03.686386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1446/2862] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:03.696453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4267575-b285-4f33-9cb7-196ddf367521 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:03.757833 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4267575-b285-4f33-9cb7-196ddf367521 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:03.762352 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b795e6-3c23-4189-9d96-b3a8825ee810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:03.762942 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b795e6-3c23-4189-9d96-b3a8825ee810 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:03.763210 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b795e6-3c23-4189-9d96-b3a8825ee810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:03.763473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b795e6-3c23-4189-9d96-b3a8825ee810 admin admin] PATH_INFO: `/v3/projects` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:03.764114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b795e6-3c23-4189-9d96-b3a8825ee810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:03.766686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b795e6-3c23-4189-9d96-b3a8825ee810 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:03.847987 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1417/2863] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:55:03 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:55:03.852028 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fba79-1553-47fa-9faa-bff8b1222202 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:03.862709 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380fba79-1553-47fa-9faa-bff8b1222202 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:03.888417 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:03.919116 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134ebca-5dc8-41ad-8e96-b3e7c58be0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:03.919730 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134ebca-5dc8-41ad-8e96-b3e7c58be0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:03.920005 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134ebca-5dc8-41ad-8e96-b3e7c58be0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:03.920270 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134ebca-5dc8-41ad-8e96-b3e7c58be0a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:03.920900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134ebca-5dc8-41ad-8e96-b3e7c58be0a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:03.923043 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134ebca-5dc8-41ad-8e96-b3e7c58be0a8 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:04.004389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1447/2864] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:55:03 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:55:04.008263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b795e6-3c23-4189-9d96-b3a8825ee810 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:04.018795 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b795e6-3c23-4189-9d96-b3a8825ee810 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:04.023378 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa1877-6d35-4b2b-958b-ea7af036c921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:04.023967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa1877-6d35-4b2b-958b-ea7af036c921 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:04.024231 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa1877-6d35-4b2b-958b-ea7af036c921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:04.024498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa1877-6d35-4b2b-958b-ea7af036c921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:04.025126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa1877-6d35-4b2b-958b-ea7af036c921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:04.026730 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa1877-6d35-4b2b-958b-ea7af036c921 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:04.038653 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1418/2865] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:55:04.042889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134ebca-5dc8-41ad-8e96-b3e7c58be0a8 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:04.053583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134ebca-5dc8-41ad-8e96-b3e7c58be0a8 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:04.058175 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378ee23-6f00-4995-b171-8627eda6a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:04.058822 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378ee23-6f00-4995-b171-8627eda6a85a admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:04.059125 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378ee23-6f00-4995-b171-8627eda6a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:04.059411 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378ee23-6f00-4995-b171-8627eda6a85a admin admin] PATH_INFO: `/v3/projects/5d233360393e474992108ffd5d6eb1e5/users/43087523999e4522b77debbb27ca492e/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:04.074884 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378ee23-6f00-4995-b171-8627eda6a85a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d233360393e474992108ffd5d6eb1e5, user_id=43087523999e4522b77debbb27ca492e, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:04.077487 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378ee23-6f00-4995-b171-8627eda6a85a admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:04.136546 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1448/2866] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:55:04 2022] PUT /identity/v3/projects/5d233360393e474992108ffd5d6eb1e5/users/43087523999e4522b77debbb27ca492e/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:04.141071 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa1877-6d35-4b2b-958b-ea7af036c921 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:04.151102 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa1877-6d35-4b2b-958b-ea7af036c921 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:04.206700 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c930c-e4a6-4639-b55f-b747b4f67de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:04.207300 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c930c-e4a6-4639-b55f-b747b4f67de4 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:04.207568 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c930c-e4a6-4639-b55f-b747b4f67de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:04.207835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c930c-e4a6-4639-b55f-b747b4f67de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:04.208512 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c930c-e4a6-4639-b55f-b747b4f67de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:04.210110 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c930c-e4a6-4639-b55f-b747b4f67de4 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:04.220581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1419/2867] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:55:04.224848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378ee23-6f00-4995-b171-8627eda6a85a admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:04.235816 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378ee23-6f00-4995-b171-8627eda6a85a admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:04.240480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b945b4-3705-4065-8735-1c5855bde9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:04.241119 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b945b4-3705-4065-8735-1c5855bde9d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:04.241386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b945b4-3705-4065-8735-1c5855bde9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:04.241650 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b945b4-3705-4065-8735-1c5855bde9d0 admin admin] PATH_INFO: `/v3/projects/5d233360393e474992108ffd5d6eb1e5/users/43087523999e4522b77debbb27ca492e/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:04.243541 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b945b4-3705-4065-8735-1c5855bde9d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d233360393e474992108ffd5d6eb1e5, user_id=43087523999e4522b77debbb27ca492e, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:04.246264 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b945b4-3705-4065-8735-1c5855bde9d0 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:04.256409 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1449/2868] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:55:04 2022] PUT /identity/v3/projects/5d233360393e474992108ffd5d6eb1e5/users/43087523999e4522b77debbb27ca492e/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:04.268816 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9f80d-a63c-44de-8c6d-db556a7599ae None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:04.269098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9f80d-a63c-44de-8c6d-db556a7599ae None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:04.269369 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9f80d-a63c-44de-8c6d-db556a7599ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:04.278284 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:04.298983 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-07f9f80d-a63c-44de-8c6d-db556a7599ae None None] MFA Rules not processed for user `43087523999e4522b77debbb27ca492e`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:04.300199 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-07f9f80d-a63c-44de-8c6d-db556a7599ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:04.367640 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1420/2869] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:04.379613 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b945b4-3705-4065-8735-1c5855bde9d0 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:04.381310 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e0b945b4-3705-4065-8735-1c5855bde9d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:04.439131 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b945b4-3705-4065-8735-1c5855bde9d0 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:04.442773 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045e4f42-a900-4a2f-b971-551d56692bfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:04.443359 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e4f42-a900-4a2f-b971-551d56692bfd service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:04.443657 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e4f42-a900-4a2f-b971-551d56692bfd service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:04.443921 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045e4f42-a900-4a2f-b971-551d56692bfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:04.452758 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e4f42-a900-4a2f-b971-551d56692bfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:04.454374 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045e4f42-a900-4a2f-b971-551d56692bfd service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:04.463596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1450/2870] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:55:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:55:04.529555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9f80d-a63c-44de-8c6d-db556a7599ae None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:04.591835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9f80d-a63c-44de-8c6d-db556a7599ae None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:04.597375 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:04.598052 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:04.598433 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:04.598651 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] PATH_INFO: `/v3/users/532932a6351d48f0a2669203e7c9317f` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:04.599873 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=532932a6351d48f0a2669203e7c9317f)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:04.601767 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:04.767637 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '532932a6351d48f0a2669203e7c9317f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:04.768297 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '532932a6351d48f0a2669203e7c9317f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:04.844501 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '532932a6351d48f0a2669203e7c9317f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:04.851879 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '532932a6351d48f0a2669203e7c9317f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:04.852498 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '532932a6351d48f0a2669203e7c9317f'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:04.869861 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1421/2871] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:55:04 2022] DELETE /identity/v3/users/532932a6351d48f0a2669203e7c9317f => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:04.874472 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c70761-51a2-42d1-a9a5-fb05270002d2 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:04.874754 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c70761-51a2-42d1-a9a5-fb05270002d2 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:04.875552 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c70761-51a2-42d1-a9a5-fb05270002d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:04.894457 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-e7c70761-51a2-42d1-a9a5-fb05270002d2 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:04.895633 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e7c70761-51a2-42d1-a9a5-fb05270002d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:04.955579 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1451/2872] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:04.965141 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:04.966624 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:04.977469 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:05.024600 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82827c53-4106-4ab5-bc56-1a541b6dd8c2 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:05.028259 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca886342-28b7-47da-b52a-2a89b4f943f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:05.028851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca886342-28b7-47da-b52a-2a89b4f943f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:05.029126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca886342-28b7-47da-b52a-2a89b4f943f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:05.029426 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca886342-28b7-47da-b52a-2a89b4f943f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:05.039092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca886342-28b7-47da-b52a-2a89b4f943f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:05.040493 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca886342-28b7-47da-b52a-2a89b4f943f9 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:05.050611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1422/2873] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:55:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:55:05.477242 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c70761-51a2-42d1-a9a5-fb05270002d2 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:05.478808 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e7c70761-51a2-42d1-a9a5-fb05270002d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:05.493084 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c70761-51a2-42d1-a9a5-fb05270002d2 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:05.498423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:05.499064 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:05.499327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:05.499623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] PATH_INFO: `/v3/projects/f1458b9e14044c5cb4546079ed3b0bf4` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:05.500958 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1458b9e14044c5cb4546079ed3b0bf4)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:05.502877 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:05.523684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:05.641118 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1458b9e14044c5cb4546079ed3b0bf4', 'request_id': 'req-27c3d8c8-4747-42f1-850d-eba65904c6d2'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:05.646658 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1458b9e14044c5cb4546079ed3b0bf4', 'request_id': 'req-27c3d8c8-4747-42f1-850d-eba65904c6d2'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:05.653202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1458b9e14044c5cb4546079ed3b0bf4', 'request_id': 'req-27c3d8c8-4747-42f1-850d-eba65904c6d2'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:05.749340 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] The token cache is being invalidate because project f1458b9e14044c5cb4546079ed3b0bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:55:05.749558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:05.750576 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:05.754052 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1452/2874] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:55:05 2022] DELETE /identity/v3/projects/f1458b9e14044c5cb4546079ed3b0bf4 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:05.756528 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca886342-28b7-47da-b52a-2a89b4f943f9 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:05.767008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca886342-28b7-47da-b52a-2a89b4f943f9 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:05.821807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e3f73-3b51-481e-ac6c-1bbd66da753c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:05.822485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e3f73-3b51-481e-ac6c-1bbd66da753c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:05.822735 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e3f73-3b51-481e-ac6c-1bbd66da753c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:05.822995 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e3f73-3b51-481e-ac6c-1bbd66da753c admin admin] PATH_INFO: `/v3/users/43087523999e4522b77debbb27ca492e` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:05.824086 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e3f73-3b51-481e-ac6c-1bbd66da753c admin admin] RBAC: Authorizing `identity:delete_user(user_id=43087523999e4522b77debbb27ca492e)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:05.825991 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e3f73-3b51-481e-ac6c-1bbd66da753c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:06.072002 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fb2e3f73-3b51-481e-ac6c-1bbd66da753c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43087523999e4522b77debbb27ca492e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:06.072581 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fb2e3f73-3b51-481e-ac6c-1bbd66da753c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43087523999e4522b77debbb27ca492e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:06.148939 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fb2e3f73-3b51-481e-ac6c-1bbd66da753c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43087523999e4522b77debbb27ca492e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:06.156576 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fb2e3f73-3b51-481e-ac6c-1bbd66da753c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43087523999e4522b77debbb27ca492e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:06.159053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-fb2e3f73-3b51-481e-ac6c-1bbd66da753c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43087523999e4522b77debbb27ca492e'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:06.175181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1423/2875] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:55:05 2022] DELETE /identity/v3/users/43087523999e4522b77debbb27ca492e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:06.578907 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:06.580327 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:06.593236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3d8c8-4747-42f1-850d-eba65904c6d2 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:06.650720 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355b435-9b03-4b24-9ff3-ff77f27d023e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:06.651347 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355b435-9b03-4b24-9ff3-ff77f27d023e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:06.651623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355b435-9b03-4b24-9ff3-ff77f27d023e admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:06.651889 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355b435-9b03-4b24-9ff3-ff77f27d023e admin admin] PATH_INFO: `/v3/projects/5d233360393e474992108ffd5d6eb1e5` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:06.652996 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355b435-9b03-4b24-9ff3-ff77f27d023e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d233360393e474992108ffd5d6eb1e5)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:06.654954 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355b435-9b03-4b24-9ff3-ff77f27d023e admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:06.755075 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3355b435-9b03-4b24-9ff3-ff77f27d023e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d233360393e474992108ffd5d6eb1e5', 'request_id': 'req-3355b435-9b03-4b24-9ff3-ff77f27d023e'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:06.760820 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3355b435-9b03-4b24-9ff3-ff77f27d023e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d233360393e474992108ffd5d6eb1e5', 'request_id': 'req-3355b435-9b03-4b24-9ff3-ff77f27d023e'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:06.767189 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3355b435-9b03-4b24-9ff3-ff77f27d023e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d233360393e474992108ffd5d6eb1e5', 'request_id': 'req-3355b435-9b03-4b24-9ff3-ff77f27d023e'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:06.835281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3355b435-9b03-4b24-9ff3-ff77f27d023e admin admin] The token cache is being invalidate because project 5d233360393e474992108ffd5d6eb1e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:55:06.835492 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3355b435-9b03-4b24-9ff3-ff77f27d023e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:06.836106 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-3355b435-9b03-4b24-9ff3-ff77f27d023e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:06.837611 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1453/2876] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:55:06 2022] DELETE /identity/v3/projects/5d233360393e474992108ffd5d6eb1e5 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:06.842658 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586ad23-21e0-4bd8-bb85-aa4c1aa6229b None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:06.843153 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586ad23-21e0-4bd8-bb85-aa4c1aa6229b None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:06.843272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586ad23-21e0-4bd8-bb85-aa4c1aa6229b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:06.862732 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-0586ad23-21e0-4bd8-bb85-aa4c1aa6229b None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:06.863917 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-0586ad23-21e0-4bd8-bb85-aa4c1aa6229b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:06.888989 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:06.922186 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1424/2877] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:06.929814 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbde7bb-1d37-4df1-8662-ee91ba4deb65 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:06.930188 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbde7bb-1d37-4df1-8662-ee91ba4deb65 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:06.930583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbde7bb-1d37-4df1-8662-ee91ba4deb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:06.950948 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-afbde7bb-1d37-4df1-8662-ee91ba4deb65 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:06.952139 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-afbde7bb-1d37-4df1-8662-ee91ba4deb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:06.961288 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1454/2878] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:06.965538 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586ad23-21e0-4bd8-bb85-aa4c1aa6229b None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:06.975544 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586ad23-21e0-4bd8-bb85-aa4c1aa6229b None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:06.980074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba2bcc-38fa-4843-a2f5-f5f8a96d1a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:06.980670 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba2bcc-38fa-4843-a2f5-f5f8a96d1a46 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:06.981140 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba2bcc-38fa-4843-a2f5-f5f8a96d1a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:06.981571 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba2bcc-38fa-4843-a2f5-f5f8a96d1a46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:06.982535 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba2bcc-38fa-4843-a2f5-f5f8a96d1a46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:55:06.982834 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba2bcc-38fa-4843-a2f5-f5f8a96d1a46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:06.984529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba2bcc-38fa-4843-a2f5-f5f8a96d1a46 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:06.995856 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1425/2879] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 16 05:55:07.000873 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbde7bb-1d37-4df1-8662-ee91ba4deb65 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:07.010271 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbde7bb-1d37-4df1-8662-ee91ba4deb65 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:07.014994 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a47fe-bceb-4885-8f68-107ab3211aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:07.016395 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a47fe-bceb-4885-8f68-107ab3211aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:07.016764 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a47fe-bceb-4885-8f68-107ab3211aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:07.017183 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a47fe-bceb-4885-8f68-107ab3211aa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:07.017798 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a47fe-bceb-4885-8f68-107ab3211aa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:07.019745 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a47fe-bceb-4885-8f68-107ab3211aa9 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:07.116471 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1455/2880] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:55:06 2022] POST /identity/v3/projects => generated 391 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:55:07.124142 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba2bcc-38fa-4843-a2f5-f5f8a96d1a46 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:07.131177 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba2bcc-38fa-4843-a2f5-f5f8a96d1a46 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:07.188098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c7970-c5b9-4e63-a07c-53e151db74b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:07.188869 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c7970-c5b9-4e63-a07c-53e151db74b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:07.189143 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c7970-c5b9-4e63-a07c-53e151db74b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:07.189408 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c7970-c5b9-4e63-a07c-53e151db74b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:07.190056 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c7970-c5b9-4e63-a07c-53e151db74b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:07.191946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c7970-c5b9-4e63-a07c-53e151db74b4 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:07.298663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1426/2881] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:55:07 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:55:07.302872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a47fe-bceb-4885-8f68-107ab3211aa9 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:07.313945 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a47fe-bceb-4885-8f68-107ab3211aa9 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:07.319092 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0148f99-5aeb-4213-ae53-5030981f737f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:07.319699 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0148f99-5aeb-4213-ae53-5030981f737f admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:07.319971 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0148f99-5aeb-4213-ae53-5030981f737f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:07.320927 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0148f99-5aeb-4213-ae53-5030981f737f admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:07.321583 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0148f99-5aeb-4213-ae53-5030981f737f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:07.323203 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0148f99-5aeb-4213-ae53-5030981f737f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:07.325482 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:07.334262 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1456/2882] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:55:07.338350 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c7970-c5b9-4e63-a07c-53e151db74b4 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:07.348819 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c7970-c5b9-4e63-a07c-53e151db74b4 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:07.353463 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2083f1-62df-4f56-b6bd-63cd9c6244b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:07.354074 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2083f1-62df-4f56-b6bd-63cd9c6244b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:07.354349 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2083f1-62df-4f56-b6bd-63cd9c6244b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:07.354615 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2083f1-62df-4f56-b6bd-63cd9c6244b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:07.355236 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2083f1-62df-4f56-b6bd-63cd9c6244b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:07.356807 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2083f1-62df-4f56-b6bd-63cd9c6244b3 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:07.371394 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1427/2883] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:55:07.374282 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0148f99-5aeb-4213-ae53-5030981f737f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:07.384890 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0148f99-5aeb-4213-ae53-5030981f737f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:07.389997 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6040b72-2f2f-4656-a276-905342ae29bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:07.390666 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6040b72-2f2f-4656-a276-905342ae29bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:07.390937 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6040b72-2f2f-4656-a276-905342ae29bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:07.391202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6040b72-2f2f-4656-a276-905342ae29bc admin admin] PATH_INFO: `/v3/projects/f66208964fc844e5ba9ea974dbc230ec/users/54d1dbea645448c0856ef451cce8e7bb/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:07.406352 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6040b72-2f2f-4656-a276-905342ae29bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66208964fc844e5ba9ea974dbc230ec, user_id=54d1dbea645448c0856ef451cce8e7bb, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:07.409075 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6040b72-2f2f-4656-a276-905342ae29bc admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:07.475109 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1457/2884] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:55:07 2022] PUT /identity/v3/projects/f66208964fc844e5ba9ea974dbc230ec/users/54d1dbea645448c0856ef451cce8e7bb/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:07.486235 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedddf8-0407-4609-b95d-aad07d390d3d None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:07.486519 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedddf8-0407-4609-b95d-aad07d390d3d None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:07.486790 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedddf8-0407-4609-b95d-aad07d390d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:07.514324 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-bfedddf8-0407-4609-b95d-aad07d390d3d None None] MFA Rules not processed for user `54d1dbea645448c0856ef451cce8e7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:07.515552 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-bfedddf8-0407-4609-b95d-aad07d390d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:07.579965 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1428/2885] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:55:07.584604 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6040b72-2f2f-4656-a276-905342ae29bc admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:07.648973 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6040b72-2f2f-4656-a276-905342ae29bc admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:07.653667 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc6af6-c266-4a74-bf40-8796c161ee46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:07.654372 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc6af6-c266-4a74-bf40-8796c161ee46 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:07.654714 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc6af6-c266-4a74-bf40-8796c161ee46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:07.655050 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc6af6-c266-4a74-bf40-8796c161ee46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:07.655751 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc6af6-c266-4a74-bf40-8796c161ee46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:07.658415 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc6af6-c266-4a74-bf40-8796c161ee46 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:07.752034 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1458/2886] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:55:07 2022] POST /identity/v3/projects => generated 391 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:55:07.755906 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedddf8-0407-4609-b95d-aad07d390d3d None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:07.767351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedddf8-0407-4609-b95d-aad07d390d3d None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:07.795015 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:07.828525 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07343b2-faed-4d78-b5c8-125c2335b733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:07.829123 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07343b2-faed-4d78-b5c8-125c2335b733 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:07.829395 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07343b2-faed-4d78-b5c8-125c2335b733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:07.829664 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07343b2-faed-4d78-b5c8-125c2335b733 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:07.830316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07343b2-faed-4d78-b5c8-125c2335b733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:07.832511 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07343b2-faed-4d78-b5c8-125c2335b733 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:07.925077 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1429/2887] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:55:07 2022] POST /identity/v3/users => generated 302 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 16 05:55:07.929179 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc6af6-c266-4a74-bf40-8796c161ee46 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:07.940002 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc6af6-c266-4a74-bf40-8796c161ee46 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:07.944933 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d79c9a-bff6-4ec4-9a06-9867bcf6e2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:07.946713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d79c9a-bff6-4ec4-9a06-9867bcf6e2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:07.946713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d79c9a-bff6-4ec4-9a06-9867bcf6e2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:07.946713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d79c9a-bff6-4ec4-9a06-9867bcf6e2b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:07.947024 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d79c9a-bff6-4ec4-9a06-9867bcf6e2b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:07.948389 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d79c9a-bff6-4ec4-9a06-9867bcf6e2b4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:07.960616 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1459/2888] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:55:07.964800 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07343b2-faed-4d78-b5c8-125c2335b733 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:07.975428 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07343b2-faed-4d78-b5c8-125c2335b733 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:07.980088 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73935d-2bb0-48a6-8125-0f00268e678f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:07.980733 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73935d-2bb0-48a6-8125-0f00268e678f admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:07.981010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73935d-2bb0-48a6-8125-0f00268e678f admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:07.981280 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73935d-2bb0-48a6-8125-0f00268e678f admin admin] PATH_INFO: `/v3/projects/d236b03c68384923b1f704d2ed41cf72/users/d5c890264bdc40eb9f1a6f849dad8e9c/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:07.995990 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73935d-2bb0-48a6-8125-0f00268e678f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d236b03c68384923b1f704d2ed41cf72, user_id=d5c890264bdc40eb9f1a6f849dad8e9c, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:07.998666 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73935d-2bb0-48a6-8125-0f00268e678f admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:08.068602 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1430/2889] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:55:07 2022] PUT /identity/v3/projects/d236b03c68384923b1f704d2ed41cf72/users/d5c890264bdc40eb9f1a6f849dad8e9c/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:08.072520 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d79c9a-bff6-4ec4-9a06-9867bcf6e2b4 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:08.082972 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d79c9a-bff6-4ec4-9a06-9867bcf6e2b4 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:08.137453 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a04c0-651b-430c-ba00-ed504f5e3ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:08.138085 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a04c0-651b-430c-ba00-ed504f5e3ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:08.138361 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a04c0-651b-430c-ba00-ed504f5e3ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:08.138628 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a04c0-651b-430c-ba00-ed504f5e3ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:08.139281 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a04c0-651b-430c-ba00-ed504f5e3ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:08.141443 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a04c0-651b-430c-ba00-ed504f5e3ef1 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:08.151009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1460/2890] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:55:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:55:08.155125 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73935d-2bb0-48a6-8125-0f00268e678f admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:08.165942 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73935d-2bb0-48a6-8125-0f00268e678f admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:08.170739 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70382e3a-0c99-43da-958a-1e176e5f519d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:08.171376 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70382e3a-0c99-43da-958a-1e176e5f519d admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:08.171647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70382e3a-0c99-43da-958a-1e176e5f519d admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:08.172163 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70382e3a-0c99-43da-958a-1e176e5f519d admin admin] PATH_INFO: `/v3/projects/d236b03c68384923b1f704d2ed41cf72/users/d5c890264bdc40eb9f1a6f849dad8e9c/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:08.173852 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70382e3a-0c99-43da-958a-1e176e5f519d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d236b03c68384923b1f704d2ed41cf72, user_id=d5c890264bdc40eb9f1a6f849dad8e9c, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:08.176409 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70382e3a-0c99-43da-958a-1e176e5f519d admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:08.186370 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1431/2891] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:55:08 2022] PUT /identity/v3/projects/d236b03c68384923b1f704d2ed41cf72/users/d5c890264bdc40eb9f1a6f849dad8e9c/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:08.197348 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c8a75-80a3-4525-946e-cc0cdd127863 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:08.197630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c8a75-80a3-4525-946e-cc0cdd127863 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:08.197942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c8a75-80a3-4525-946e-cc0cdd127863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:08.227440 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-bb6c8a75-80a3-4525-946e-cc0cdd127863 None None] MFA Rules not processed for user `d5c890264bdc40eb9f1a6f849dad8e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:08.228658 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-bb6c8a75-80a3-4525-946e-cc0cdd127863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:08.260617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:08.297185 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1461/2892] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:08 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:08.307276 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70382e3a-0c99-43da-958a-1e176e5f519d admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:08.308946 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-70382e3a-0c99-43da-958a-1e176e5f519d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:08.367312 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70382e3a-0c99-43da-958a-1e176e5f519d admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:08.371016 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80087e-7b3d-423c-8624-be77991a2c54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:08.371614 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80087e-7b3d-423c-8624-be77991a2c54 service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:08.371884 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80087e-7b3d-423c-8624-be77991a2c54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:08.372148 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80087e-7b3d-423c-8624-be77991a2c54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:08.382532 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80087e-7b3d-423c-8624-be77991a2c54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:08.383949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80087e-7b3d-423c-8624-be77991a2c54 service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:08.394691 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1432/2893] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:55:08 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:55:08.443555 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c8a75-80a3-4525-946e-cc0cdd127863 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:08.444935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-bb6c8a75-80a3-4525-946e-cc0cdd127863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:08.457511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c8a75-80a3-4525-946e-cc0cdd127863 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:08.461174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b293d-b9e7-4a49-8b83-e3612479206f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:08.461767 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b293d-b9e7-4a49-8b83-e3612479206f service cinder] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:08.462083 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b293d-b9e7-4a49-8b83-e3612479206f service cinder] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:08.462351 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b293d-b9e7-4a49-8b83-e3612479206f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:08.517942 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b293d-b9e7-4a49-8b83-e3612479206f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:08.519358 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b293d-b9e7-4a49-8b83-e3612479206f service cinder] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:08.528055 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1462/2894] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:55:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:55:13.228881 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80087e-7b3d-423c-8624-be77991a2c54 service cinder] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:13.297077 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80087e-7b3d-423c-8624-be77991a2c54 service cinder] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:13.301656 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:13.302323 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:13.302599 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:13.302865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] PATH_INFO: `/v3/users/54d1dbea645448c0856ef451cce8e7bb` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:13.304048 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54d1dbea645448c0856ef451cce8e7bb)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:13.305977 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:13.333516 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:13.535201 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54d1dbea645448c0856ef451cce8e7bb'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:13.535847 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54d1dbea645448c0856ef451cce8e7bb'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:13.624115 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54d1dbea645448c0856ef451cce8e7bb'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:13.634042 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54d1dbea645448c0856ef451cce8e7bb'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:13.635604 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54d1dbea645448c0856ef451cce8e7bb'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:13.657466 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1433/2895] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:55:13 2022] DELETE /identity/v3/users/54d1dbea645448c0856ef451cce8e7bb => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:13.662253 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a7773-6259-4dac-9659-ec7ded87a4a1 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:13.662695 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a7773-6259-4dac-9659-ec7ded87a4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:13.662888 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a7773-6259-4dac-9659-ec7ded87a4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:13.688011 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-d41a7773-6259-4dac-9659-ec7ded87a4a1 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:13.692393 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d41a7773-6259-4dac-9659-ec7ded87a4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:13.769605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1463/2896] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:13.779601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:13.781097 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:13.838351 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0420ad7-c619-447c-b616-c909ef11a415 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:13.842079 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bb7ea-c6c2-465c-907a-b35fdabef917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:13.842682 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bb7ea-c6c2-465c-907a-b35fdabef917 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:13.842956 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bb7ea-c6c2-465c-907a-b35fdabef917 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:13.843222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bb7ea-c6c2-465c-907a-b35fdabef917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:13.852857 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bb7ea-c6c2-465c-907a-b35fdabef917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:13.854342 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bb7ea-c6c2-465c-907a-b35fdabef917 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:13.864978 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1434/2897] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:55:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:55:14.282388 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a7773-6259-4dac-9659-ec7ded87a4a1 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:14.286846 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-d41a7773-6259-4dac-9659-ec7ded87a4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:14.297053 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a7773-6259-4dac-9659-ec7ded87a4a1 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:14.301617 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:14.302276 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:14.302548 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:14.302811 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4 admin admin] PATH_INFO: `/v3/projects/f66208964fc844e5ba9ea974dbc230ec` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:14.304368 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f66208964fc844e5ba9ea974dbc230ec)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:14.305968 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:14.412508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f66208964fc844e5ba9ea974dbc230ec', 'request_id': 'req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:14.418318 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f66208964fc844e5ba9ea974dbc230ec', 'request_id': 'req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:14.424404 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f66208964fc844e5ba9ea974dbc230ec', 'request_id': 'req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:14.426060 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:14.479935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4 admin admin] The token cache is being invalidate because project f66208964fc844e5ba9ea974dbc230ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:55:14.480319 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:14.480935 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-c29b2b4e-cb5f-4d5f-95d2-15848fd98ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:14.482537 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1464/2898] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:55:14 2022] DELETE /identity/v3/projects/f66208964fc844e5ba9ea974dbc230ec => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:55:14.487151 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bb7ea-c6c2-465c-907a-b35fdabef917 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:14.496673 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bb7ea-c6c2-465c-907a-b35fdabef917 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:14.561343 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:14.561918 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:14.562217 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:14.562490 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] PATH_INFO: `/v3/users/d5c890264bdc40eb9f1a6f849dad8e9c` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:14.563609 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5c890264bdc40eb9f1a6f849dad8e9c)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:14.565527 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:14.762498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43826882-fd26-4c76-a07e-bb830d612b6e None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:14.763004 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43826882-fd26-4c76-a07e-bb830d612b6e None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:14.763162 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43826882-fd26-4c76-a07e-bb830d612b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:14.763799 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:14.766326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5c890264bdc40eb9f1a6f849dad8e9c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:14.767200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5c890264bdc40eb9f1a6f849dad8e9c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:14.782838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-43826882-fd26-4c76-a07e-bb830d612b6e None None] MFA Rules not processed for user `e6147d4c0d534cb2b580fda7fc2272ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:14.784052 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-43826882-fd26-4c76-a07e-bb830d612b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:14.812356 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5c890264bdc40eb9f1a6f849dad8e9c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:14.820450 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5c890264bdc40eb9f1a6f849dad8e9c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:14.829083 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5c890264bdc40eb9f1a6f849dad8e9c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:14.839901 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1435/2899] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:55:14 2022] DELETE /identity/v3/users/d5c890264bdc40eb9f1a6f849dad8e9c => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:14.846810 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1465/2900] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:55:15.226227 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:15.227703 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:15.284981 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446167ea-8472-40e7-808a-6abc6ea0dba2 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:15.288607 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414050b-974a-4476-b6dc-b9fbc2cdb444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:15.289197 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414050b-974a-4476-b6dc-b9fbc2cdb444 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:15.289477 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414050b-974a-4476-b6dc-b9fbc2cdb444 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:15.289738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414050b-974a-4476-b6dc-b9fbc2cdb444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:15.291425 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3414050b-974a-4476-b6dc-b9fbc2cdb444 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:15.301754 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414050b-974a-4476-b6dc-b9fbc2cdb444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:15.303181 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414050b-974a-4476-b6dc-b9fbc2cdb444 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:15.311273 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1436/2901] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:55:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:55:15.322231 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43826882-fd26-4c76-a07e-bb830d612b6e None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:15.323678 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-43826882-fd26-4c76-a07e-bb830d612b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:15.340045 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43826882-fd26-4c76-a07e-bb830d612b6e None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:15.401337 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605816d9-fc25-453b-a7c9-3b2aa7cc5373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:15.402009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605816d9-fc25-453b-a7c9-3b2aa7cc5373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:15.402286 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605816d9-fc25-453b-a7c9-3b2aa7cc5373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:15.402558 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605816d9-fc25-453b-a7c9-3b2aa7cc5373 admin admin] PATH_INFO: `/v3/projects/d236b03c68384923b1f704d2ed41cf72` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:15.403779 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605816d9-fc25-453b-a7c9-3b2aa7cc5373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d236b03c68384923b1f704d2ed41cf72)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:15.405674 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605816d9-fc25-453b-a7c9-3b2aa7cc5373 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:15.499450 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-605816d9-fc25-453b-a7c9-3b2aa7cc5373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd236b03c68384923b1f704d2ed41cf72', 'request_id': 'req-605816d9-fc25-453b-a7c9-3b2aa7cc5373'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:15.505690 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-605816d9-fc25-453b-a7c9-3b2aa7cc5373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd236b03c68384923b1f704d2ed41cf72', 'request_id': 'req-605816d9-fc25-453b-a7c9-3b2aa7cc5373'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:15.512138 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-605816d9-fc25-453b-a7c9-3b2aa7cc5373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd236b03c68384923b1f704d2ed41cf72', 'request_id': 'req-605816d9-fc25-453b-a7c9-3b2aa7cc5373'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:15.595160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-605816d9-fc25-453b-a7c9-3b2aa7cc5373 admin admin] The token cache is being invalidate because project d236b03c68384923b1f704d2ed41cf72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:55:15.595507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-605816d9-fc25-453b-a7c9-3b2aa7cc5373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:15.600512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-605816d9-fc25-453b-a7c9-3b2aa7cc5373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:15.600512 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1466/2902] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:55:15 2022] DELETE /identity/v3/projects/d236b03c68384923b1f704d2ed41cf72 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:15.602898 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f88be9-af7d-4f7e-80e9-41196bbe4242 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:15.603384 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f88be9-af7d-4f7e-80e9-41196bbe4242 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:15.603611 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f88be9-af7d-4f7e-80e9-41196bbe4242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:15.624068 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-52f88be9-af7d-4f7e-80e9-41196bbe4242 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:15.626106 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-52f88be9-af7d-4f7e-80e9-41196bbe4242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:15.688542 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1437/2903] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:15.696788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733ca27-18c1-40de-a467-6f96805c8a18 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:15.696788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733ca27-18c1-40de-a467-6f96805c8a18 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:15.696788 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733ca27-18c1-40de-a467-6f96805c8a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:15.719022 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-3733ca27-18c1-40de-a467-6f96805c8a18 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:15.720206 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-3733ca27-18c1-40de-a467-6f96805c8a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:15.728221 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1467/2904] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:15.732694 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f88be9-af7d-4f7e-80e9-41196bbe4242 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:15.743262 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f88be9-af7d-4f7e-80e9-41196bbe4242 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:15.747749 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432111c4-414b-4c75-bd9e-21bd384dda38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:15.748347 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432111c4-414b-4c75-bd9e-21bd384dda38 admin admin] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:15.748619 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432111c4-414b-4c75-bd9e-21bd384dda38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:15.748884 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432111c4-414b-4c75-bd9e-21bd384dda38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:15.749522 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432111c4-414b-4c75-bd9e-21bd384dda38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81556) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 16 05:55:15.750660 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432111c4-414b-4c75-bd9e-21bd384dda38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:15.752258 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432111c4-414b-4c75-bd9e-21bd384dda38 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:15.767058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1438/2905] 10.222.0.49 () {58 vars in 1153 bytes} [Fri Sep 16 05:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 16 05:55:15.768525 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733ca27-18c1-40de-a467-6f96805c8a18 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:15.777668 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733ca27-18c1-40de-a467-6f96805c8a18 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:15.783124 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546e3fe-d537-4972-8e07-93e8dd0808c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:15.783708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546e3fe-d537-4972-8e07-93e8dd0808c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:15.783979 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546e3fe-d537-4972-8e07-93e8dd0808c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:15.784239 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546e3fe-d537-4972-8e07-93e8dd0808c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:15.784859 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546e3fe-d537-4972-8e07-93e8dd0808c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:15.786786 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546e3fe-d537-4972-8e07-93e8dd0808c1 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:15.852466 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:15.859480 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1468/2906] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:55:15 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:55:15.864026 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432111c4-414b-4c75-bd9e-21bd384dda38 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:15.874245 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432111c4-414b-4c75-bd9e-21bd384dda38 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:15.928574 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:15.938431 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7874a9-d9ee-4952-baeb-321fcb0d4f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:15.939105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7874a9-d9ee-4952-baeb-321fcb0d4f97 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:15.939380 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7874a9-d9ee-4952-baeb-321fcb0d4f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:15.939643 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7874a9-d9ee-4952-baeb-321fcb0d4f97 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:15.940309 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7874a9-d9ee-4952-baeb-321fcb0d4f97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:15.942319 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7874a9-d9ee-4952-baeb-321fcb0d4f97 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:16.035204 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1439/2907] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:55:15 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:55:16.040140 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546e3fe-d537-4972-8e07-93e8dd0808c1 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:16.051779 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546e3fe-d537-4972-8e07-93e8dd0808c1 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:16.056871 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08588f8-3b27-461e-a837-53232f201a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:16.057536 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08588f8-3b27-461e-a837-53232f201a8c admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:16.057714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7874a9-d9ee-4952-baeb-321fcb0d4f97 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:16.059529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08588f8-3b27-461e-a837-53232f201a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:16.059529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08588f8-3b27-461e-a837-53232f201a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:16.059529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08588f8-3b27-461e-a837-53232f201a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:16.060916 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08588f8-3b27-461e-a837-53232f201a8c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:16.066565 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-9e7874a9-d9ee-4952-baeb-321fcb0d4f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:16.072872 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1469/2908] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:55:16.076903 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08588f8-3b27-461e-a837-53232f201a8c admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:16.090431 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08588f8-3b27-461e-a837-53232f201a8c admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:16.096944 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85899844-51ee-46a7-8e5a-7e5e7debe12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:16.097513 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85899844-51ee-46a7-8e5a-7e5e7debe12c admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:16.097910 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85899844-51ee-46a7-8e5a-7e5e7debe12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:16.098538 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85899844-51ee-46a7-8e5a-7e5e7debe12c admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:16.098917 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85899844-51ee-46a7-8e5a-7e5e7debe12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:16.100936 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85899844-51ee-46a7-8e5a-7e5e7debe12c admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:16.116166 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1470/2909] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:55:16.125334 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85899844-51ee-46a7-8e5a-7e5e7debe12c admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:16.135464 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85899844-51ee-46a7-8e5a-7e5e7debe12c admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:16.140355 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c9b70-ac09-4385-bc4a-fb8169e54782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:16.141350 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c9b70-ac09-4385-bc4a-fb8169e54782 admin admin] REQUEST_METHOD: `PUT` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:16.141703 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c9b70-ac09-4385-bc4a-fb8169e54782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:16.141999 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c9b70-ac09-4385-bc4a-fb8169e54782 admin admin] PATH_INFO: `/v3/projects/cd6ff35ff4aa47c6b5f6bd07efe6fb02/users/77fa87466c9a49df9ae8a0fdbcb14557/roles/413cc5f8baab440bab827a3547b65f33` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:16.144935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7874a9-d9ee-4952-baeb-321fcb0d4f97 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:16.150113 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13150d06-59d2-4d1a-a298-4425742fc55f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9975c0dab76f4ccbb4d2107e8e780ae5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:16.150714 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13150d06-59d2-4d1a-a298-4425742fc55f service placement] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:16.151008 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13150d06-59d2-4d1a-a298-4425742fc55f service placement] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:16.151271 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13150d06-59d2-4d1a-a298-4425742fc55f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:16.152976 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-13150d06-59d2-4d1a-a298-4425742fc55f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:16.157713 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c9b70-ac09-4385-bc4a-fb8169e54782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd6ff35ff4aa47c6b5f6bd07efe6fb02, user_id=77fa87466c9a49df9ae8a0fdbcb14557, role_id=413cc5f8baab440bab827a3547b65f33)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:16.160394 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c9b70-ac09-4385-bc4a-fb8169e54782 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:16.164835 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13150d06-59d2-4d1a-a298-4425742fc55f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:16.166442 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13150d06-59d2-4d1a-a298-4425742fc55f service placement] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:16.177657 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1440/2910] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:55:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:55:16.278924 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1471/2911] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:55:16 2022] PUT /identity/v3/projects/cd6ff35ff4aa47c6b5f6bd07efe6fb02/users/77fa87466c9a49df9ae8a0fdbcb14557/roles/413cc5f8baab440bab827a3547b65f33 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:55:16.290526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8434b-0c28-49c5-9b9e-8bd59c7fb223 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:16.290804 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8434b-0c28-49c5-9b9e-8bd59c7fb223 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:16.291069 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8434b-0c28-49c5-9b9e-8bd59c7fb223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:16.324718 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-68b8434b-0c28-49c5-9b9e-8bd59c7fb223 None None] MFA Rules not processed for user `77fa87466c9a49df9ae8a0fdbcb14557`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:16.325934 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-68b8434b-0c28-49c5-9b9e-8bd59c7fb223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:16.391189 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1441/2912] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:16.395596 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c9b70-ac09-4385-bc4a-fb8169e54782 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:16.457875 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c9b70-ac09-4385-bc4a-fb8169e54782 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:16.462607 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd971cd-9e91-4269-b305-588c45cfa590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:16.463209 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd971cd-9e91-4269-b305-588c45cfa590 admin admin] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:16.463476 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd971cd-9e91-4269-b305-588c45cfa590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:16.463739 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd971cd-9e91-4269-b305-588c45cfa590 admin admin] PATH_INFO: `/v3/projects` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:16.464357 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd971cd-9e91-4269-b305-588c45cfa590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:16.466292 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd971cd-9e91-4269-b305-588c45cfa590 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:16.569201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1472/2913] 10.222.0.49 () {60 vars in 1153 bytes} [Fri Sep 16 05:55:16 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:55:16.570411 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8434b-0c28-49c5-9b9e-8bd59c7fb223 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:16.582896 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8434b-0c28-49c5-9b9e-8bd59c7fb223 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:16.638853 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95576e-21f4-44c7-bd87-b1cc68baaac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:16.639521 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95576e-21f4-44c7-bd87-b1cc68baaac1 admin admin] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:16.639707 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95576e-21f4-44c7-bd87-b1cc68baaac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:16.639967 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95576e-21f4-44c7-bd87-b1cc68baaac1 admin admin] PATH_INFO: `/v3/users` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:16.640698 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95576e-21f4-44c7-bd87-b1cc68baaac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:16.642659 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95576e-21f4-44c7-bd87-b1cc68baaac1 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:16.651662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:16.753332 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1442/2914] 10.222.0.49 () {60 vars in 1144 bytes} [Fri Sep 16 05:55:16 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 16 05:55:16.757336 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd971cd-9e91-4269-b305-588c45cfa590 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:16.767847 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd971cd-9e91-4269-b305-588c45cfa590 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:16.772510 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e869ea-1e89-4266-9a91-d415a0b8b0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:16.773110 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e869ea-1e89-4266-9a91-d415a0b8b0cf admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:16.773386 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e869ea-1e89-4266-9a91-d415a0b8b0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:16.773652 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e869ea-1e89-4266-9a91-d415a0b8b0cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:16.774382 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e869ea-1e89-4266-9a91-d415a0b8b0cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:16.775963 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e869ea-1e89-4266-9a91-d415a0b8b0cf admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:16.787076 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1473/2915] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:55:16.794490 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95576e-21f4-44c7-bd87-b1cc68baaac1 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:16.804865 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95576e-21f4-44c7-bd87-b1cc68baaac1 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:16.809556 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7c057-28b4-4067-b391-8bb3703e1433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:16.810215 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7c057-28b4-4067-b391-8bb3703e1433 admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:16.810601 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7c057-28b4-4067-b391-8bb3703e1433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:16.810867 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7c057-28b4-4067-b391-8bb3703e1433 admin admin] PATH_INFO: `/v3/projects/944e55e7db8c4f56a90791e7ddf56392/users/cd00ffaf76714c86bf45fccf98c7ad75/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:16.826416 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7c057-28b4-4067-b391-8bb3703e1433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=944e55e7db8c4f56a90791e7ddf56392, user_id=cd00ffaf76714c86bf45fccf98c7ad75, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:16.829777 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7c057-28b4-4067-b391-8bb3703e1433 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:16.922216 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1443/2916] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:55:16 2022] PUT /identity/v3/projects/944e55e7db8c4f56a90791e7ddf56392/users/cd00ffaf76714c86bf45fccf98c7ad75/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:55:16.926295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e869ea-1e89-4266-9a91-d415a0b8b0cf admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:16.936156 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e869ea-1e89-4266-9a91-d415a0b8b0cf admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:16.992817 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efb6a6-fa2a-4c85-86f1-46cdbc52c0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:16.993448 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efb6a6-fa2a-4c85-86f1-46cdbc52c0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:16.993722 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efb6a6-fa2a-4c85-86f1-46cdbc52c0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:16.994012 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efb6a6-fa2a-4c85-86f1-46cdbc52c0f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:16.994641 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efb6a6-fa2a-4c85-86f1-46cdbc52c0f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:16.996215 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efb6a6-fa2a-4c85-86f1-46cdbc52c0f1 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:17.006681 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1474/2917] 10.222.0.49 () {58 vars in 1122 bytes} [Fri Sep 16 05:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 16 05:55:17.010902 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7c057-28b4-4067-b391-8bb3703e1433 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:17.021418 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7c057-28b4-4067-b391-8bb3703e1433 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:17.026094 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aeb4f9-5787-4a1a-843d-d0e11f4cc44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:17.026750 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aeb4f9-5787-4a1a-843d-d0e11f4cc44a admin admin] REQUEST_METHOD: `PUT` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:17.027028 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aeb4f9-5787-4a1a-843d-d0e11f4cc44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:17.027303 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aeb4f9-5787-4a1a-843d-d0e11f4cc44a admin admin] PATH_INFO: `/v3/projects/944e55e7db8c4f56a90791e7ddf56392/users/cd00ffaf76714c86bf45fccf98c7ad75/roles/9c3e3861a9ce4828829f793b31a275f0` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:17.029209 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aeb4f9-5787-4a1a-843d-d0e11f4cc44a admin admin] RBAC: Authorizing `identity:create_grant(project_id=944e55e7db8c4f56a90791e7ddf56392, user_id=cd00ffaf76714c86bf45fccf98c7ad75, role_id=9c3e3861a9ce4828829f793b31a275f0)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:17.031938 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aeb4f9-5787-4a1a-843d-d0e11f4cc44a admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:17.042244 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1444/2918] 10.222.0.49 () {60 vars in 1483 bytes} [Fri Sep 16 05:55:17 2022] PUT /identity/v3/projects/944e55e7db8c4f56a90791e7ddf56392/users/cd00ffaf76714c86bf45fccf98c7ad75/roles/9c3e3861a9ce4828829f793b31a275f0 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:55:17.053148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854aa9f-2372-49a5-9798-f4de39b51505 None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:17.053432 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854aa9f-2372-49a5-9798-f4de39b51505 None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:17.056148 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8854aa9f-2372-49a5-9798-f4de39b51505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:17.083287 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-8854aa9f-2372-49a5-9798-f4de39b51505 None None] MFA Rules not processed for user `cd00ffaf76714c86bf45fccf98c7ad75`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:17.084502 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8854aa9f-2372-49a5-9798-f4de39b51505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:17.093752 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:17.155550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1475/2919] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:55:17.169458 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aeb4f9-5787-4a1a-843d-d0e11f4cc44a admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:17.172737 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-22aeb4f9-5787-4a1a-843d-d0e11f4cc44a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:17.233339 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aeb4f9-5787-4a1a-843d-d0e11f4cc44a admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:17.237645 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c7555-4535-4645-8723-632cdc2486bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff50e50587254bac912fb342a7e09891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:17.240075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c7555-4535-4645-8723-632cdc2486bf service cinder] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:17.240075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c7555-4535-4645-8723-632cdc2486bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:17.240075 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c7555-4535-4645-8723-632cdc2486bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:17.247937 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c7555-4535-4645-8723-632cdc2486bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:17.249388 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c7555-4535-4645-8723-632cdc2486bf service cinder] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:17.260629 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1445/2920] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:55:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:55:17.764478 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854aa9f-2372-49a5-9798-f4de39b51505 None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:17.825766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8854aa9f-2372-49a5-9798-f4de39b51505 None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:17.830670 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:17.831303 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:17.831577 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:17.831848 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] PATH_INFO: `/v3/users/77fa87466c9a49df9ae8a0fdbcb14557` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:17.833003 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77fa87466c9a49df9ae8a0fdbcb14557)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:17.834984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:18.048605 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77fa87466c9a49df9ae8a0fdbcb14557'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:18.049174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77fa87466c9a49df9ae8a0fdbcb14557'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:18.120498 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77fa87466c9a49df9ae8a0fdbcb14557'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:18.121337 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77fa87466c9a49df9ae8a0fdbcb14557'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:18.128971 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77fa87466c9a49df9ae8a0fdbcb14557'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:18.146295 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1476/2921] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:55:17 2022] DELETE /identity/v3/users/77fa87466c9a49df9ae8a0fdbcb14557 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:18.150738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acf8de-a4e3-457f-b6c6-7f29b7448087 None None] REQUEST_METHOD: `POST` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:18.151281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acf8de-a4e3-457f-b6c6-7f29b7448087 None None] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:18.151281 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4acf8de-a4e3-457f-b6c6-7f29b7448087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:18.170783 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.auth.core [None req-f4acf8de-a4e3-457f-b6c6-7f29b7448087 None None] MFA Rules not processed for user `2ebd28e621914177981b750fe6a7566b`. Rule list: `[]` (Enabled: `True`). {{(pid=81556) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:55:18.172543 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f4acf8de-a4e3-457f-b6c6-7f29b7448087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:18.231272 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1446/2922] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 16 05:55:18.241322 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:18.242960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:18.276007 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:18.307126 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dbf6bc-c700-4e08-934f-de9050008993 admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:18.311476 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216406f2-288d-458f-8239-2f6f04d970b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:18.312202 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216406f2-288d-458f-8239-2f6f04d970b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:18.312802 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216406f2-288d-458f-8239-2f6f04d970b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:18.313071 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216406f2-288d-458f-8239-2f6f04d970b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:18.323117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216406f2-288d-458f-8239-2f6f04d970b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:18.324550 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216406f2-288d-458f-8239-2f6f04d970b7 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:18.334723 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1477/2923] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:55:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:55:18.882741 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acf8de-a4e3-457f-b6c6-7f29b7448087 None None] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:18.884326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-f4acf8de-a4e3-457f-b6c6-7f29b7448087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:18.898331 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4acf8de-a4e3-457f-b6c6-7f29b7448087 None None] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:18.902983 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:18.903608 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:18.903880 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:18.904147 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] PATH_INFO: `/v3/projects/cd6ff35ff4aa47c6b5f6bd07efe6fb02` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:18.905275 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd6ff35ff4aa47c6b5f6bd07efe6fb02)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:18.907461 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:19.044849 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd6ff35ff4aa47c6b5f6bd07efe6fb02', 'request_id': 'req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:19.047663 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:19.051547 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd6ff35ff4aa47c6b5f6bd07efe6fb02', 'request_id': 'req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:19.057540 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd6ff35ff4aa47c6b5f6bd07efe6fb02', 'request_id': 'req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:19.150392 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] The token cache is being invalidate because project cd6ff35ff4aa47c6b5f6bd07efe6fb02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:55:19.150721 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:19.151310 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:19.152811 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1447/2924] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:55:18 2022] DELETE /identity/v3/projects/cd6ff35ff4aa47c6b5f6bd07efe6fb02 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:19.157413 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216406f2-288d-458f-8239-2f6f04d970b7 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:19.166960 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216406f2-288d-458f-8239-2f6f04d970b7 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:19.223086 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:19.223712 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:19.223989 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:19.224256 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] PATH_INFO: `/v3/users/cd00ffaf76714c86bf45fccf98c7ad75` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:19.225364 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd00ffaf76714c86bf45fccf98c7ad75)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:19.227298 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:19.396900 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd00ffaf76714c86bf45fccf98c7ad75'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:19.397934 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd00ffaf76714c86bf45fccf98c7ad75'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:19.526408 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd00ffaf76714c86bf45fccf98c7ad75'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:19.527174 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd00ffaf76714c86bf45fccf98c7ad75'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:19.541869 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd00ffaf76714c86bf45fccf98c7ad75'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:19.580647 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1478/2925] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:55:19 2022] DELETE /identity/v3/users/cd00ffaf76714c86bf45fccf98c7ad75 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:20.158647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:20.158647 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:20.177360 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d18cf-d0c4-48dc-868d-2c72eb3b4948 admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:20.242016 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:20.242642 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:20.242914 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:20.243179 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] PATH_INFO: `/v3/projects/944e55e7db8c4f56a90791e7ddf56392` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:20.244293 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] RBAC: Authorizing `identity:delete_project(project_id=944e55e7db8c4f56a90791e7ddf56392)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:20.246444 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:20.364326 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '944e55e7db8c4f56a90791e7ddf56392', 'request_id': 'req-86a025cf-74bf-45c7-b67c-8743461ca29c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:20.370406 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '944e55e7db8c4f56a90791e7ddf56392', 'request_id': 'req-86a025cf-74bf-45c7-b67c-8743461ca29c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:20.382188 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '944e55e7db8c4f56a90791e7ddf56392', 'request_id': 'req-86a025cf-74bf-45c7-b67c-8743461ca29c'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:20.458392 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] The token cache is being invalidate because project 944e55e7db8c4f56a90791e7ddf56392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81556) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:55:20.458679 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:20.459294 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:55:20.460775 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1448/2926] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:55:20 2022] DELETE /identity/v3/projects/944e55e7db8c4f56a90791e7ddf56392 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:55:29.575332 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=67,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:30.455058 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:30.786778 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:30.788529 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:30.993787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0222a8a-0a98-4b41-82a0-de5696112bcd admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:31.053957 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c2e9e-2984-43b6-9bbe-34c8ed240795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:31.054507 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c2e9e-2984-43b6-9bbe-34c8ed240795 service nova] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:31.054780 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c2e9e-2984-43b6-9bbe-34c8ed240795 service nova] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:31.055042 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c2e9e-2984-43b6-9bbe-34c8ed240795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:31.056766 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-231c2e9e-2984-43b6-9bbe-34c8ed240795 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:31.069212 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c2e9e-2984-43b6-9bbe-34c8ed240795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:31.070805 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231c2e9e-2984-43b6-9bbe-34c8ed240795 service nova] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:31.082088 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1479/2927] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:55:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:55:41.076785 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=62 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:55:52.635963 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:55:52.637355 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:52.707456 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a025cf-74bf-45c7-b67c-8743461ca29c admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:55:52.712686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04a60c-15fc-4d8d-88b7-7ece60e27dd9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9975c0dab76f4ccbb4d2107e8e780ae5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:55:52.713286 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04a60c-15fc-4d8d-88b7-7ece60e27dd9 service placement] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:55:52.713559 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04a60c-15fc-4d8d-88b7-7ece60e27dd9 service placement] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:55:52.713851 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04a60c-15fc-4d8d-88b7-7ece60e27dd9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:55:52.715537 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5a04a60c-15fc-4d8d-88b7-7ece60e27dd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:55:52.728327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04a60c-15fc-4d8d-88b7-7ece60e27dd9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:55:52.729908 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04a60c-15fc-4d8d-88b7-7ece60e27dd9 service placement] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:55:52.740105 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1449/2928] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:55:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:56:02.735404 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=32 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:57:10.710116 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913b7dd-6e9b-4488-a1a3-add881af69bd None None] REQUEST_METHOD: `POST` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:57:10.710693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913b7dd-6e9b-4488-a1a3-add881af69bd None None] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:57:10.710693 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913b7dd-6e9b-4488-a1a3-add881af69bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:57:10.729407 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.auth.core [None req-e913b7dd-6e9b-4488-a1a3-add881af69bd None None] MFA Rules not processed for user `e6147d4c0d534cb2b580fda7fc2272ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81557) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 16 05:57:10.730623 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-e913b7dd-6e9b-4488-a1a3-add881af69bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:57:10.786335 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1480/2929] 10.222.0.49 () {58 vars in 958 bytes} [Fri Sep 16 05:57:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 16 05:57:11.121613 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04a60c-15fc-4d8d-88b7-7ece60e27dd9 service placement] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:57:11.123092 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-5a04a60c-15fc-4d8d-88b7-7ece60e27dd9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:57:11.178762 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04a60c-15fc-4d8d-88b7-7ece60e27dd9 service placement] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:57:11.182327 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc640b-e52f-4a2b-8360-463b70bdba08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:57:11.182949 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc640b-e52f-4a2b-8360-463b70bdba08 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:57:11.183222 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc640b-e52f-4a2b-8360-463b70bdba08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:57:11.183490 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc640b-e52f-4a2b-8360-463b70bdba08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:57:11.185203 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-38fc640b-e52f-4a2b-8360-463b70bdba08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:57:11.195481 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc640b-e52f-4a2b-8360-463b70bdba08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:57:11.196903 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc640b-e52f-4a2b-8360-463b70bdba08 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:57:11.205499 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1450/2930] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:57:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:57:20.779427 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=25 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:57:21.201191 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=26 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:58:58.969009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913b7dd-6e9b-4488-a1a3-add881af69bd None None] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:58:58.979160 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913b7dd-6e9b-4488-a1a3-add881af69bd None None] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:58:58.982670 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6413d-35bd-48e3-a6bf-98722f0987c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:58:58.983308 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6413d-35bd-48e3-a6bf-98722f0987c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:58:58.983578 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6413d-35bd-48e3-a6bf-98722f0987c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:58:58.983838 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6413d-35bd-48e3-a6bf-98722f0987c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:58:58.985497 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-60e6413d-35bd-48e3-a6bf-98722f0987c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:58:58.995878 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6413d-35bd-48e3-a6bf-98722f0987c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:58:58.997251 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6413d-35bd-48e3-a6bf-98722f0987c1 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:58:59.006320 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1481/2931] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:58:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:59:00.197488 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc640b-e52f-4a2b-8360-463b70bdba08 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:59:00.198975 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-38fc640b-e52f-4a2b-8360-463b70bdba08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:59:00.216555 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc640b-e52f-4a2b-8360-463b70bdba08 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:59:00.222348 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56776b1-df3d-48c3-bd55-5d43f238af47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7db75c20e9744feb8b5dc78c554f4977', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:59:00.222953 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56776b1-df3d-48c3-bd55-5d43f238af47 service nova] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:59:00.223219 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56776b1-df3d-48c3-bd55-5d43f238af47 service nova] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:59:00.223485 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56776b1-df3d-48c3-bd55-5d43f238af47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:59:00.226087 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-d56776b1-df3d-48c3-bd55-5d43f238af47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:59:00.241175 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56776b1-df3d-48c3-bd55-5d43f238af47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:59:00.242772 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56776b1-df3d-48c3-bd55-5d43f238af47 service nova] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:59:00.256725 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1451/2932] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:59:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:59:00.353756 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6413d-35bd-48e3-a6bf-98722f0987c1 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:59:00.362274 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6413d-35bd-48e3-a6bf-98722f0987c1 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:59:00.366998 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fadb8-28f2-4a34-8097-544690f6ae06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:59:00.367630 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fadb8-28f2-4a34-8097-544690f6ae06 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:59:00.367930 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fadb8-28f2-4a34-8097-544690f6ae06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:59:00.368196 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512fadb8-28f2-4a34-8097-544690f6ae06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:59:00.370070 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-512fadb8-28f2-4a34-8097-544690f6ae06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:59:00.382090 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fadb8-28f2-4a34-8097-544690f6ae06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:59:00.383511 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512fadb8-28f2-4a34-8097-544690f6ae06 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:59:00.392819 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1482/2933] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:59:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:59:00.628212 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56776b1-df3d-48c3-bd55-5d43f238af47 service nova] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:59:00.638715 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56776b1-df3d-48c3-bd55-5d43f238af47 service nova] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:59:00.642361 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bdcc6-485b-49f8-9e2a-6ce44b3da38d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:59:00.642986 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bdcc6-485b-49f8-9e2a-6ce44b3da38d service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:59:00.643261 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bdcc6-485b-49f8-9e2a-6ce44b3da38d service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:59:00.643526 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1bdcc6-485b-49f8-9e2a-6ce44b3da38d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:59:00.645237 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-3a1bdcc6-485b-49f8-9e2a-6ce44b3da38d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:59:00.664587 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1bdcc6-485b-49f8-9e2a-6ce44b3da38d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:59:00.666112 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1bdcc6-485b-49f8-9e2a-6ce44b3da38d service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:59:00.678453 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1452/2934] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:59:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:59:01.867116 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fadb8-28f2-4a34-8097-544690f6ae06 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:59:01.868669 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-512fadb8-28f2-4a34-8097-544690f6ae06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:59:01.878984 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512fadb8-28f2-4a34-8097-544690f6ae06 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:59:01.882913 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e525da-a71b-4ada-80c7-5966f054c942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:59:01.883580 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e525da-a71b-4ada-80c7-5966f054c942 service neutron] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:59:01.883852 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e525da-a71b-4ada-80c7-5966f054c942 service neutron] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:59:01.884117 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e525da-a71b-4ada-80c7-5966f054c942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:59:01.885750 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-89e525da-a71b-4ada-80c7-5966f054c942 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:59:01.899423 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e525da-a71b-4ada-80c7-5966f054c942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:59:01.900823 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e525da-a71b-4ada-80c7-5966f054c942 service neutron] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:59:01.912967 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1483/2935] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:59:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:59:10.672744 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=12 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:59:11.906784 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=18 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:59:13.479813 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bdcc6-485b-49f8-9e2a-6ce44b3da38d service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:59:13.488594 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1bdcc6-485b-49f8-9e2a-6ce44b3da38d service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:59:13.493126 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61337e-71b9-4ca2-b26c-18fea12b87f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e923be9aeed43728690d24d85dbe062', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:59:13.493738 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61337e-71b9-4ca2-b26c-18fea12b87f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:59:13.494045 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61337e-71b9-4ca2-b26c-18fea12b87f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:59:13.494316 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61337e-71b9-4ca2-b26c-18fea12b87f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:59:13.507496 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2f61337e-71b9-4ca2-b26c-18fea12b87f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:59:13.578489 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61337e-71b9-4ca2-b26c-18fea12b87f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:59:13.580053 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61337e-71b9-4ca2-b26c-18fea12b87f2 service neutron] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:59:13.589839 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1453/2936] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 05:59:17.137955 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e525da-a71b-4ada-80c7-5966f054c942 service neutron] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:59:17.150832 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e525da-a71b-4ada-80c7-5966f054c942 service neutron] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:59:17.158201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a772080-382c-4fba-a3a4-e1a2ace89faa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9975c0dab76f4ccbb4d2107e8e780ae5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:59:17.158201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a772080-382c-4fba-a3a4-e1a2ace89faa service placement] REQUEST_METHOD: `GET` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:59:17.158201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a772080-382c-4fba-a3a4-e1a2ace89faa service placement] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:59:17.158201 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a772080-382c-4fba-a3a4-e1a2ace89faa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:59:17.159488 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-6a772080-382c-4fba-a3a4-e1a2ace89faa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:59:17.171999 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a772080-382c-4fba-a3a4-e1a2ace89faa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:59:17.173567 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a772080-382c-4fba-a3a4-e1a2ace89faa service placement] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:59:17.184566 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1484/2937] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 05:59:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 16 05:59:20.837969 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61337e-71b9-4ca2-b26c-18fea12b87f2 service neutron] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:59:20.839371 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-2f61337e-71b9-4ca2-b26c-18fea12b87f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:59:20.886355 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61337e-71b9-4ca2-b26c-18fea12b87f2 service neutron] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:59:20.890855 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:59:20.891476 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:59:20.891745 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:59:20.892010 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] PATH_INFO: `/v3/users/e6147d4c0d534cb2b580fda7fc2272ad` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:59:20.893114 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6147d4c0d534cb2b580fda7fc2272ad)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:59:20.895043 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:59:21.031007 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6147d4c0d534cb2b580fda7fc2272ad'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.031662 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6147d4c0d534cb2b580fda7fc2272ad'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.084894 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6147d4c0d534cb2b580fda7fc2272ad'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.093907 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6147d4c0d534cb2b580fda7fc2272ad'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.094529 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6147d4c0d534cb2b580fda7fc2272ad'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.112473 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1454/2938] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:59:20 2022] DELETE /identity/v3/users/e6147d4c0d534cb2b580fda7fc2272ad => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:59:21.366554 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a772080-382c-4fba-a3a4-e1a2ace89faa service placement] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:59:21.412824 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a772080-382c-4fba-a3a4-e1a2ace89faa service placement] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:59:21.472077 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:59:21.472708 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:59:21.473044 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:59:21.473236 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] PATH_INFO: `/v3/projects/22b431b5f9f344a6b72c3ee49d1c29ce` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:59:21.474515 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=22b431b5f9f344a6b72c3ee49d1c29ce)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:59:21.476349 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:59:21.576939 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22b431b5f9f344a6b72c3ee49d1c29ce', 'request_id': 'req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.582523 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22b431b5f9f344a6b72c3ee49d1c29ce', 'request_id': 'req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.588920 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22b431b5f9f344a6b72c3ee49d1c29ce', 'request_id': 'req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.662119 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] The token cache is being invalidate because project 22b431b5f9f344a6b72c3ee49d1c29ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:59:21.662481 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.663351 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.665125 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1485/2939] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:59:21 2022] DELETE /identity/v3/projects/22b431b5f9f344a6b72c3ee49d1c29ce => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:59:21.669317 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:59:21.671157 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:59:21.684897 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9412bf6-3e46-4263-9c19-3269d85664dd admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:59:21.715486 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:59:21.741782 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:59:21.742941 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:59:21.743263 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:59:21.743532 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] PATH_INFO: `/v3/users/a122b9caa0414fc1bad0dbf5c7b86220` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:59:21.744671 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a122b9caa0414fc1bad0dbf5c7b86220)` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:59:21.746590 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:59:21.902961 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a122b9caa0414fc1bad0dbf5c7b86220'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.903631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a122b9caa0414fc1bad0dbf5c7b86220'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.969631 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a122b9caa0414fc1bad0dbf5c7b86220'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.977784 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a122b9caa0414fc1bad0dbf5c7b86220'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.978454 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.notifications [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a122b9caa0414fc1bad0dbf5c7b86220'} {{(pid=81556) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:21.996617 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1455/2940] 10.222.0.49 () {58 vars in 1224 bytes} [Fri Sep 16 05:59:21 2022] DELETE /identity/v3/users/a122b9caa0414fc1bad0dbf5c7b86220 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 16 05:59:22.247097 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] Authenticating user token {{(pid=81557) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 05:59:22.248508 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.fernet_utils [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81557) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 05:59:22.261672 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e1d9b-de6a-460a-bc2e-a7d67194fb9f admin admin] Validating token access rules against request {{(pid=81557) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 05:59:22.312134 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:59:22.320521 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ebd28e621914177981b750fe6a7566b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c317c5acb304c5d983c73b9ed1b6802', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81557) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 05:59:22.321136 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 05:59:22.321405 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 05:59:22.321670 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d admin admin] PATH_INFO: `/v3/projects/c8ecac97ea8c4737a02c0a7c145b4b91` {{(pid=81557) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 05:59:22.322787 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ecac97ea8c4737a02c0a7c145b4b91)` {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 05:59:22.324707 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d admin admin] RBAC: Authorization granted {{(pid=81557) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 05:59:22.422343 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ecac97ea8c4737a02c0a7c145b4b91', 'request_id': 'req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:22.427796 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ecac97ea8c4737a02c0a7c145b4b91', 'request_id': 'req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:22.434009 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ecac97ea8c4737a02c0a7c145b4b91', 'request_id': 'req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d'} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:22.512671 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d admin admin] The token cache is being invalidate because project c8ecac97ea8c4737a02c0a7c145b4b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81557) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 16 05:59:22.512892 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:22.513684 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG keystone.notifications [None req-df7b99da-fdd5-4778-a5e9-638ec4bb8f6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81557) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 16 05:59:22.515257 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: [pid: 81557|app: 0|req: 1486/2941] 10.222.0.49 () {58 vars in 1233 bytes} [Fri Sep 16 05:59:22 2022] DELETE /identity/v3/projects/c8ecac97ea8c4737a02c0a7c145b4b91 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 16 05:59:31.993111 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=38,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 05:59:32.509670 dv-cinder-857528-3-iscsi devstack@keystone.service[81557]: DEBUG dbcounter [-] [81557] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81557) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 16 06:00:59.585338 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] Authenticating user token {{(pid=81556) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 16 06:00:59.586759 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 06:00:59.696686 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f7f0c-4f8e-4803-8b51-71a82d63eefc admin admin] Validating token access rules against request {{(pid=81556) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 16 06:00:59.757036 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62289d-8fef-481e-9099-f6e523615a6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9975c0dab76f4ccbb4d2107e8e780ae5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b2e27932984de79ee8dfbe95312ac2', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81556) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 16 06:00:59.757634 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62289d-8fef-481e-9099-f6e523615a6c service placement] REQUEST_METHOD: `GET` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 16 06:00:59.757935 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62289d-8fef-481e-9099-f6e523615a6c service placement] SCRIPT_NAME: `/identity` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 16 06:00:59.758200 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62289d-8fef-481e-9099-f6e523615a6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81556) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 16 06:00:59.759934 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.fernet_utils [None req-df62289d-8fef-481e-9099-f6e523615a6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81556) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 16 06:00:59.771809 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62289d-8fef-481e-9099-f6e523615a6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 16 06:00:59.773398 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62289d-8fef-481e-9099-f6e523615a6c service placement] RBAC: Authorization granted {{(pid=81556) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 16 06:00:59.783291 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: [pid: 81556|app: 0|req: 1456/2942] 10.222.0.49 () {60 vars in 1365 bytes} [Fri Sep 16 06:00:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 16 06:01:09.779098 dv-cinder-857528-3-iscsi devstack@keystone.service[81556]: DEBUG dbcounter [-] [81556] Writing DB stats keystone:SELECT=74 {{(pid=81556) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}